2023-11-24T06:09:36.709 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T06:09:36.710 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_rishabh@teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} --name rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558 -- /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/orig.config.yaml 2023-11-24T06:09:36.753 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:09:36.837 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} email: null first_in_suite: false job_id: '7465558' kernel: flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3305 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMBWw4LPwMQvxVPgW6Z5nJVeimE42KGyRzyRz7kIiXdPqs+1FRULDPryrnIqWH/SVfFlOo5bSnajqLLE+P57dUU= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWKYd2+qmve4xjdoRtS92UFJvCTKl8gtKVkGgl5I4mgJOj2juuxiBXVDTfPPdQh3E8RHK5Ef3PVr73pU1bGv8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - check-counter: counters: mgr: - min: 4 name: finisher-volumes.complete_latency.avgcount - expected_val: 0 name: finisher-volumes.queue_len - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestPerModuleFinsherThread teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_05:41:33 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T06:09:36.838 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa; will attempt to use it 2023-11-24T06:09:36.838 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks 2023-11-24T06:09:36.838 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T06:09:36.840 INFO:teuthology.task.internal:Checking packages... 2023-11-24T06:09:36.863 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T06:09:36.864 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T06:09:36.864 INFO:teuthology.packaging:ref: None 2023-11-24T06:09:36.864 INFO:teuthology.packaging:tag: None 2023-11-24T06:09:36.864 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T06:09:36.864 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:09:36.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T06:09:37.008 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T06:09:37.008 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T06:09:37.014 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T06:09:37.014 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T06:09:37.023 INFO:teuthology.task.internal:Saving configuration 2023-11-24T06:09:37.039 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T06:09:37.047 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:09:37.074 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:04:13.066076', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMBWw4LPwMQvxVPgW6Z5nJVeimE42KGyRzyRz7kIiXdPqs+1FRULDPryrnIqWH/SVfFlOo5bSnajqLLE+P57dUU='} 2023-11-24T06:09:37.096 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:04:13.067676', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWKYd2+qmve4xjdoRtS92UFJvCTKl8gtKVkGgl5I4mgJOj2juuxiBXVDTfPPdQh3E8RHK5Ef3PVr73pU1bGv8='} 2023-11-24T06:09:37.096 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T06:09:37.103 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T06:09:37.103 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T06:09:37.103 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T06:09:37.173 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6cb15d5b80>, signals=[15]) 2023-11-24T06:09:37.174 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T06:09:37.181 INFO:teuthology.task.internal:Opening connections... 2023-11-24T06:09:37.181 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:09:37.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:09:37.280 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:09:37.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:09:37.361 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T06:09:37.368 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2023-11-24T06:09:37.385 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-11-24T06:09:37.386 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2023-11-24T06:09:37.442 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2023-11-24T06:09:37.442 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2023-11-24T06:09:37.442 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T06:09:37.443 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T06:09:37.444 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-11-24T06:09:37.469 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2023-11-24T06:09:37.486 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2023-11-24T06:09:37.486 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2023-11-24T06:09:37.543 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T06:09:37.544 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T06:09:37.545 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T06:09:37.545 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-11-24T06:09:37.568 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T06:09:37.579 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T06:09:37.586 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T06:09:37.589 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2023-11-24T06:09:37.592 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2023-11-24T06:09:37.608 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T06:09:37.619 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T06:09:37.619 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2023-11-24T06:09:37.647 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2023-11-24T06:09:37.669 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T06:09:37.724 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T06:09:37.743 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T06:09:37.743 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, timeout 300 2023-11-24T06:09:37.744 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T06:09:37.744 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T06:09:37.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:09:37.760 DEBUG:teuthology.task.kernel:sha1 for mon.a is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:09:37.760 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T06:09:37.760 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-11-24T06:09:37.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:09:37.761 DEBUG:teuthology.task.kernel:sha1 for mon.b is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:09:37.761 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T06:09:37.761 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2023-11-24T06:09:37.816 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-361.el9.x86_64 2023-11-24T06:09:37.816 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:09:37.817 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T06:09:37.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:09:37.820 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-361.el9.x86_64 2023-11-24T06:09:37.821 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:09:37.821 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T06:09:37.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:09:37.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum upgrade -y linux-firmware 2023-11-24T06:09:37.964 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum upgrade -y linux-firmware 2023-11-24T06:09:40.012 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-11-24T06:09:40.098 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-11-24T06:09:41.538 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2023-11-24T06:09:41.931 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - BaseOS 2.5 MB/s | 2.0 MB 00:00 2023-11-24T06:09:48.054 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 1.3 MB/s | 7.9 MB 00:06 2023-11-24T06:09:49.977 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 7.9 MB 00:08 2023-11-24T06:09:51.920 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 23 MB/s | 18 MB 00:00 2023-11-24T06:09:52.512 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 18 MB 00:03 2023-11-24T06:09:59.470 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 5.8 MB 00:03 2023-11-24T06:10:00.745 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 5.8 MB 00:04 2023-11-24T06:10:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2023-11-24T06:10:02.304 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2023-11-24T06:10:03.122 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2023-11-24T06:10:04.780 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 20 MB 00:02 2023-11-24T06:10:08.164 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-11-24T06:10:09.671 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-11-24T06:10:10.712 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-11-24T06:10:10.727 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 1 Package 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:10:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 328 M 2023-11-24T06:10:10.729 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-11-24T06:10:12.128 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2023-11-24T06:10:12.132 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-11-24T06:10:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2023-11-24T06:10:12.132 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 1 Package 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 328 M 2023-11-24T06:10:12.133 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2023-11-24T06:10:15.656 INFO:teuthology.orchestra.run.smithi064.stdout:linux-firmware-20231030-141.el9.noarch.rpm 67 MB/s | 328 MB 00:04 2023-11-24T06:10:15.658 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:10:15.658 INFO:teuthology.orchestra.run.smithi064.stdout:Total 67 MB/s | 328 MB 00:04 2023-11-24T06:10:15.658 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-11-24T06:10:15.760 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-11-24T06:10:15.760 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-11-24T06:10:17.180 INFO:teuthology.orchestra.run.smithi162.stdout:linux-firmware-20231030-141.el9.noarch.rpm 65 MB/s | 328 MB 00:05 2023-11-24T06:10:17.181 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:10:17.181 INFO:teuthology.orchestra.run.smithi162.stdout:Total 65 MB/s | 328 MB 00:05 2023-11-24T06:10:17.182 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2023-11-24T06:10:17.264 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2023-11-24T06:10:17.265 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-11-24T06:10:19.454 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2023-11-24T06:10:19.454 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2023-11-24T06:10:23.847 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-11-24T06:10:24.246 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:10:24.339 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:10:25.274 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:10:25.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:10:25.319 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2023-11-24T06:10:25.663 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:10:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:10:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:10:25.697 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2023-11-24T06:10:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T06:10:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:10:25.698 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-11-24T06:10:25.742 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:10:25.967 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T06:10:25.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:10:26.319 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T06:10:26.319 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T06:10:26.321 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi064.front.sepia.ceph.com to complete... 2023-11-24T06:10:26.698 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:10:26.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:10:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:10:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:10:27.114 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2023-11-24T06:10:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T06:10:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:10:27.115 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2023-11-24T06:10:27.408 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T06:10:27.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:10:27.751 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T06:10:27.752 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T06:10:27.754 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi162.front.sepia.ceph.com to complete... 2023-11-24T06:10:39.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24 06:10:39 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T06:10:39.865 INFO:teuthology.orchestra.run.smithi064.stderr:FINISHED --2023-11-24 06:10:39-- 2023-11-24T06:10:39.865 INFO:teuthology.orchestra.run.smithi064.stderr:Total wall clock time: 14s 2023-11-24T06:10:39.865 INFO:teuthology.orchestra.run.smithi064.stderr:Downloaded: 1 files, 306M in 13s (23.1 MB/s) 2023-11-24T06:10:39.866 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T06:10:39.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T06:10:41.809 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying... ######################################## 2023-11-24T06:10:41.870 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing... ######################################## 2023-11-24T06:10:41.956 INFO:teuthology.orchestra.run.smithi064.stdout:Updating / installing... 2023-11-24T06:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T06:10:52.740 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24 06:10:52 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T06:10:52.740 INFO:teuthology.orchestra.run.smithi162.stderr:FINISHED --2023-11-24 06:10:52-- 2023-11-24T06:10:52.740 INFO:teuthology.orchestra.run.smithi162.stderr:Total wall clock time: 25s 2023-11-24T06:10:52.740 INFO:teuthology.orchestra.run.smithi162.stderr:Downloaded: 1 files, 306M in 25s (12.5 MB/s) 2023-11-24T06:10:52.741 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T06:10:52.741 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T06:10:54.583 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying... ######################################## 2023-11-24T06:10:54.637 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing... ######################################## 2023-11-24T06:10:54.716 INFO:teuthology.orchestra.run.smithi162.stdout:Updating / installing... 2023-11-24T06:11:01.727 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T06:11:04.512 INFO:teuthology.orchestra.run.smithi064.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T06:11:17.116 INFO:teuthology.orchestra.run.smithi162.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T06:11:26.535 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T06:11:26.536 DEBUG:teuthology.orchestra.run.smithi064:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T06:11:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:11:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T06:11:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T06:11:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T06:11:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T06:11:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T06:11:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T06:11:26.604 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T06:11:26.605 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T06:11:26.606 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T06:11:26.607 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T06:11:26.608 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T06:11:26.609 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T06:11:26.610 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T06:11:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T06:11:26.612 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T06:11:26.612 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T06:11:26.613 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T06:11:26.614 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T06:11:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T06:11:26.616 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T06:11:26.617 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T06:11:26.618 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T06:11:26.619 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T06:11:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T06:11:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T06:11:26.622 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T06:11:26.623 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T06:11:26.624 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T06:11:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T06:11:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T06:11:26.627 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T06:11:26.628 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T06:11:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T06:11:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T06:11:26.631 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T06:11:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T06:11:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T06:11:26.634 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T06:11:26.635 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T06:11:26.636 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T06:11:26.637 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T06:11:26.638 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T06:11:26.639 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T06:11:26.640 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T06:11:26.641 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T06:11:26.642 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T06:11:26.643 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T06:11:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T06:11:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T06:11:26.646 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T06:11:26.647 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T06:11:26.648 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T06:11:26.649 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T06:11:26.650 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T06:11:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T06:11:26.652 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T06:11:26.653 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T06:11:26.654 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T06:11:26.655 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T06:11:26.656 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T06:11:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T06:11:26.658 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T06:11:26.659 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T06:11:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T06:11:26.661 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T06:11:26.662 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T06:11:26.663 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T06:11:26.664 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T06:11:26.665 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T06:11:26.666 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T06:11:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T06:11:26.668 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T06:11:26.669 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T06:11:26.670 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T06:11:26.671 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T06:11:26.672 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T06:11:26.673 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T06:11:26.674 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T06:11:26.675 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T06:11:26.676 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T06:11:26.677 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T06:11:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T06:11:26.679 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T06:11:26.680 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T06:11:26.681 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T06:11:26.682 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T06:11:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T06:11:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T06:11:26.687 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T06:11:26.691 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T06:11:26.692 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T06:11:26.692 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T06:11:26.693 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T06:11:26.694 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T06:11:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T06:11:26.701 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T06:11:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T06:11:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T06:11:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T06:11:26.706 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T06:11:26.707 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T06:11:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T06:11:26.709 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T06:11:26.710 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T06:11:26.711 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T06:11:26.712 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T06:11:26.713 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T06:11:26.714 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T06:11:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T06:11:26.716 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T06:11:26.717 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T06:11:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T06:11:26.719 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T06:11:26.720 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T06:11:26.721 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T06:11:26.722 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T06:11:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T06:11:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T06:11:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T06:11:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T06:11:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T06:11:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T06:11:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T06:11:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T06:11:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T06:11:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T06:11:26.733 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T06:11:26.734 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T06:11:26.735 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T06:11:26.736 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T06:11:26.737 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T06:11:26.738 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T06:11:26.739 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T06:11:26.740 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T06:11:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T06:11:26.742 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T06:11:26.743 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T06:11:26.744 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T06:11:26.745 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T06:11:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T06:11:26.747 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T06:11:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T06:11:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T06:11:26.750 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T06:11:26.751 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T06:11:26.752 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T06:11:26.753 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T06:11:26.754 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T06:11:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T06:11:26.756 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T06:11:26.757 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T06:11:26.758 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T06:11:26.759 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T06:11:26.760 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T06:11:26.761 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T06:11:26.762 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T06:11:26.763 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T06:11:26.764 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T06:11:26.765 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T06:11:26.765 DEBUG:teuthology.orchestra.run.smithi064:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T06:11:26.789 INFO:teuthology.orchestra.run.smithi064.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T06:11:26.789 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T06:11:26.789 INFO:teuthology.orchestra.run.smithi064.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:26.789 INFO:teuthology.orchestra.run.smithi064.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:26.789 INFO:teuthology.orchestra.run.smithi064.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T06:11:26.790 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T06:11:26.791 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-11-24T06:11:26.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2023-11-24T06:11:26.872 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2023-11-24T06:11:26.872 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2023-11-24T06:11:26.872 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 68.el9 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8264254 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2023-11-24T06:11:26.873 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:11:26.874 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T06:11:26.875 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T06:11:26.875 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 6.6.0-g8247bbb5b470 2023-11-24T06:11:26.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T06:11:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2023-11-24T06:11:28.989 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T06:11:29.019 INFO:teuthology.orchestra.run.smithi064.stderr:done 2023-11-24T06:11:29.021 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-11-24T06:11:29.076 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.o2RTXoFbt6 2023-11-24T06:11:29.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.o2RTXoFbt6 2023-11-24T06:11:29.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.o2RTXoFbt6 2023-11-24T06:11:29.296 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.o2RTXoFbt6 is 6KB 2023-11-24T06:11:29.344 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.o2RTXoFbt6 2023-11-24T06:11:29.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T06:11:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T06:11:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T06:11:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T06:11:29.427 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T06:11:29.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T06:11:29.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2023-11-24T06:11:38.264 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T06:11:38.264 DEBUG:teuthology.orchestra.run.smithi162:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T06:11:38.333 INFO:teuthology.orchestra.run.smithi162.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:11:38.333 INFO:teuthology.orchestra.run.smithi162.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T06:11:38.333 INFO:teuthology.orchestra.run.smithi162.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T06:11:38.333 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T06:11:38.333 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T06:11:38.333 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T06:11:38.333 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T06:11:38.334 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T06:11:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T06:11:38.336 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T06:11:38.337 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T06:11:38.338 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T06:11:38.339 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T06:11:38.340 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T06:11:38.341 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T06:11:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T06:11:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T06:11:38.344 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T06:11:38.345 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T06:11:38.346 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T06:11:38.347 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T06:11:38.348 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T06:11:38.349 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T06:11:38.350 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T06:11:38.351 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T06:11:38.352 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T06:11:38.353 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T06:11:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T06:11:38.355 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T06:11:38.356 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T06:11:38.357 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T06:11:38.358 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T06:11:38.359 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T06:11:38.361 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T06:11:38.362 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T06:11:38.363 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T06:11:38.365 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T06:11:38.369 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T06:11:38.374 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T06:11:38.376 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T06:11:38.377 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T06:11:38.378 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T06:11:38.379 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T06:11:38.380 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T06:11:38.381 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T06:11:38.382 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T06:11:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T06:11:38.384 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T06:11:38.385 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T06:11:38.386 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T06:11:38.387 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T06:11:38.388 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T06:11:38.389 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T06:11:38.390 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T06:11:38.391 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T06:11:38.392 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T06:11:38.393 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T06:11:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T06:11:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T06:11:38.396 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T06:11:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T06:11:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T06:11:38.399 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T06:11:38.400 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T06:11:38.401 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T06:11:38.402 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T06:11:38.403 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T06:11:38.404 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T06:11:38.405 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T06:11:38.406 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T06:11:38.407 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T06:11:38.408 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T06:11:38.409 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T06:11:38.410 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T06:11:38.411 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T06:11:38.412 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T06:11:38.413 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T06:11:38.414 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T06:11:38.415 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T06:11:38.416 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T06:11:38.417 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T06:11:38.418 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T06:11:38.419 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T06:11:38.420 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T06:11:38.421 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T06:11:38.422 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T06:11:38.423 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T06:11:38.424 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T06:11:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T06:11:38.426 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T06:11:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T06:11:38.428 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T06:11:38.429 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T06:11:38.430 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T06:11:38.431 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T06:11:38.432 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T06:11:38.433 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T06:11:38.434 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T06:11:38.435 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T06:11:38.436 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T06:11:38.437 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T06:11:38.438 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T06:11:38.439 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T06:11:38.440 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T06:11:38.441 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T06:11:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T06:11:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T06:11:38.444 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T06:11:38.445 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T06:11:38.446 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T06:11:38.447 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T06:11:38.448 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T06:11:38.449 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T06:11:38.450 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T06:11:38.451 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T06:11:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T06:11:38.453 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T06:11:38.454 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T06:11:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T06:11:38.456 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T06:11:38.457 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T06:11:38.458 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T06:11:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T06:11:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T06:11:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T06:11:38.462 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T06:11:38.463 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T06:11:38.464 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T06:11:38.465 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T06:11:38.466 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T06:11:38.468 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T06:11:38.469 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T06:11:38.471 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T06:11:38.472 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T06:11:38.474 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T06:11:38.475 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T06:11:38.478 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T06:11:38.479 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T06:11:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T06:11:38.481 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T06:11:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T06:11:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T06:11:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T06:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T06:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T06:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T06:11:38.485 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T06:11:38.485 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T06:11:38.485 DEBUG:teuthology.orchestra.run.smithi162:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:fi 2023-11-24T06:11:38.508 INFO:teuthology.orchestra.run.smithi162.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:fi 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T06:11:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:fi 2023-11-24T06:11:38.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -qi grub2-tools 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Name : grub2-tools 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Epoch : 1 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Version : 2.06 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Release : 68.el9 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: x86_64 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Group : Unspecified 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Size : 8264254 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:License : GPLv3+ 2023-11-24T06:11:38.586 INFO:teuthology.orchestra.run.smithi162.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:Packager : builder@centos.org 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:Vendor : CentOS 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:Summary : Support tools for GRUB. 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:Description : 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T06:11:38.587 INFO:teuthology.orchestra.run.smithi162.stdout:hardware devices. 2023-11-24T06:11:38.588 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:11:38.588 INFO:teuthology.orchestra.run.smithi162.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T06:11:38.588 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T06:11:38.589 INFO:teuthology.task.kernel:Updating grub on smithi162 to boot 6.6.0-g8247bbb5b470 2023-11-24T06:11:38.589 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T06:11:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2023-11-24T06:11:40.658 INFO:teuthology.orchestra.run.smithi162.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T06:11:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:done 2023-11-24T06:11:40.701 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2023-11-24T06:11:40.716 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.qdU5LDTg3v 2023-11-24T06:11:40.716 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qdU5LDTg3v 2023-11-24T06:11:40.785 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.qdU5LDTg3v 2023-11-24T06:11:40.939 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.qdU5LDTg3v is 6KB 2023-11-24T06:11:40.988 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.qdU5LDTg3v 2023-11-24T06:11:41.002 DEBUG:teuthology.orchestra.run.smithi162:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T06:11:41.068 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T06:11:41.068 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T06:11:41.068 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T06:11:41.068 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T06:11:41.069 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T06:11:41.216 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2023-11-24T06:11:59.621 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-24T06:11:59.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T06:11:59.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:11.222 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-24T06:12:11.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-24T06:12:11.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:45.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-11-24T06:12:54.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-24T06:12:54.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:59.626 DEBUG:teuthology.orchestra.remote:timed out 2023-11-24T06:13:08.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T06:13:08.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:13:16.139 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-11-24T06:13:16.823 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T06:13:16.823 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-g8247bbb5b470"... 2023-11-24T06:13:16.823 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-11-24T06:13:16.871 INFO:teuthology.orchestra.run.smithi064.stdout:6.6.0-g8247bbb5b470 2023-11-24T06:13:16.871 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T06:13:16.871 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T06:13:16.871 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T06:13:17.872 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-24T06:13:17.873 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T06:13:18.023 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2023-11-24T06:13:18.074 DEBUG:teuthology.parallel:result is None 2023-11-24T06:13:54.810 DEBUG:teuthology.orchestra.remote:timed out 2023-11-24T06:14:06.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-24T06:14:06.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:14:07.192 DEBUG:teuthology.orchestra.run.smithi162:> true 2023-11-24T06:14:07.913 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-24T06:14:07.913 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-g8247bbb5b470"... 2023-11-24T06:14:07.913 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2023-11-24T06:14:07.967 INFO:teuthology.orchestra.run.smithi162.stdout:6.6.0-g8247bbb5b470 2023-11-24T06:14:07.967 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T06:14:07.967 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T06:14:07.967 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T06:14:08.968 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-24T06:14:08.968 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T06:14:09.122 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2023-11-24T06:14:09.163 DEBUG:teuthology.parallel:result is None 2023-11-24T06:14:09.163 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T06:14:09.179 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T06:14:09.179 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:14:09.190 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:14:09.249 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T06:14:09.256 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T06:14:09.262 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T06:14:09.262 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:14:09.268 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:14:09.349 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T06:14:09.355 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T06:14:09.356 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:14:09.364 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:14:09.494 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:14:09.512 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:14:09.544 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:14:09.559 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:14:09.564 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T06:14:09.571 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T06:14:09.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:14:09.577 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:14:09.676 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T06:14:09.685 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T06:14:09.685 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:14:09.691 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:14:09.743 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:14:09.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:14:09.939 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:14:10.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:14:10.141 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:14:10.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:14:10.246 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:14:10.333 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:14:10.444 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:14:10.531 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:14:10.630 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:14:10.630 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:14:10.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2023-11-24T06:14:10.737 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2023-11-24T06:14:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:14:10.896 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:14:11.268 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T06:14:11.277 INFO:teuthology.task.internal:Starting timer... 2023-11-24T06:14:11.277 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T06:14:11.291 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T06:14:11.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2023-11-24T06:14:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2023-11-24T06:14:11.441 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2023-11-24T06:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2023-11-24T06:14:11.599 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T06:14:11.599 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2023-11-24T06:14:11.680 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2023-11-24T06:14:11.680 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2023-11-24T06:14:11.749 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2023-11-24T06:14:11.749 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2023-11-24T06:14:11.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:14:11.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:14:11.853 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:14:11.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:14:11.948 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T06:14:11.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:14:12.069 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:14:12.187 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T06:14:12.206 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-24T06:14:12.207 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T06:14:12.222 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T06:14:12.223 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T06:24:29.312 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2023-11-24T06:24:29.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T06:24:29.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:24:29.455 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-11-24T06:24:29.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T06:24:29.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-24T06:24:29.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:24:29.760 DEBUG:teuthology.orchestra.run.smithi162:> true 2023-11-24T06:24:29.913 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-24T06:24:29.913 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T06:24:29.924 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T06:24:29.924 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:24:29.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:24:29.933 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:24:29.933 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:24:30.033 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:24:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:24:30.086 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:24:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:24:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2023-11-24T06:24:30.202 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2023-11-24T06:24:30.219 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:24:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:24:30.323 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:24:30.361 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:24:30.445 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:24:30.453 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:24:30.454 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2023-11-24T06:24:30.454 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.454 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.454 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.454 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:24:30.498 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:24:30.498 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-11-24T06:24:30.498 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.498 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.498 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.498 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:24:30.499 INFO:teuthology.run_tasks:Running task install... 2023-11-24T06:24:30.509 DEBUG:teuthology.task.install:project ceph 2023-11-24T06:24:30.509 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T06:24:30.509 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T06:24:30.509 INFO:teuthology.task.install:Using flavor: default 2023-11-24T06:24:30.515 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T06:24:30.515 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T06:24:30.516 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:24:30.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:24:30.518 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:24:30.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:24:30.754 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:24:30.754 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:24:30.767 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:24:30.767 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:24:30.947 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:24:30.947 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:24:30.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:24:30.980 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:24:30.980 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:24:30.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:24:31.048 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:24:31.048 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:24:31.093 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:24:31.094 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:24:31.193 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:24:31.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:24:31.324 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:24:31.387 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:24:31.414 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2023-11-24T06:24:31.418 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2023-11-24T06:24:31.485 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2023-11-24T06:24:31.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2023-11-24T06:24:31.781 INFO:teuthology.orchestra.run.smithi162.stdout:73 files removed 2023-11-24T06:24:31.822 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:24:31.871 INFO:teuthology.orchestra.run.smithi064.stdout:73 files removed 2023-11-24T06:24:31.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:24:32.856 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 149 kB/s | 78 kB 00:00 2023-11-24T06:24:33.022 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 133 kB/s | 78 kB 00:00 2023-11-24T06:24:33.482 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 18 kB/s | 9.5 kB 00:00 2023-11-24T06:24:33.530 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2023-11-24T06:24:34.064 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2023-11-24T06:24:34.082 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2023-11-24T06:24:34.811 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - AppStream 9.8 MB/s | 6.4 MB 00:00 2023-11-24T06:24:34.917 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 8.2 MB/s | 6.4 MB 00:00 2023-11-24T06:24:36.470 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2023-11-24T06:24:36.569 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2023-11-24T06:24:38.425 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 7.9 MB 00:01 2023-11-24T06:24:41.356 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 7.9 MB 00:04 2023-11-24T06:24:41.580 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 18 MB 00:01 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 21 MB/s | 18 MB 00:00 2023-11-24T06:24:45.917 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 5.8 MB 00:00 2023-11-24T06:24:47.960 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 15 kB 00:00 2023-11-24T06:24:48.119 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 5.8 MB 00:00 2023-11-24T06:24:49.288 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2023-11-24T06:24:50.314 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 15 kB 00:00 2023-11-24T06:24:52.769 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 20 MB 00:02 2023-11-24T06:24:55.071 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-11-24T06:24:57.981 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:24:57.982 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:24:58.000 INFO:teuthology.orchestra.run.smithi162.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:24:58.001 INFO:teuthology.orchestra.run.smithi162.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:24:58.002 INFO:teuthology.orchestra.run.smithi162.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:24:58.004 INFO:teuthology.orchestra.run.smithi162.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:24:58.006 INFO:teuthology.orchestra.run.smithi162.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:24:58.009 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:24:58.011 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:24:58.089 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2023-11-24T06:24:58.119 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================ 2023-11-24T06:24:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2023-11-24T06:24:58.119 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================ 2023-11-24T06:24:58.119 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2023-11-24T06:24:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:24:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:24:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:24:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:24:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:24:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:24:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:24:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:24:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:24:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:24:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:24:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:24:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:24:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:24:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:24:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:24:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:24:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:24:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:24:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:24:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:24:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2023-11-24T06:24:58.135 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================ 2023-11-24T06:24:58.136 INFO:teuthology.orchestra.run.smithi162.stdout:Install 132 Packages 2023-11-24T06:24:58.136 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 15 Packages 2023-11-24T06:24:58.136 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:24:58.136 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 204 M 2023-11-24T06:24:58.137 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2023-11-24T06:24:58.487 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2023-11-24T06:24:58.830 INFO:teuthology.orchestra.run.smithi162.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 20 kB/s | 6.5 kB 00:00 2023-11-24T06:24:59.156 INFO:teuthology.orchestra.run.smithi162.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 2.6 MB/s | 859 kB 00:00 2023-11-24T06:24:59.222 INFO:teuthology.orchestra.run.smithi162.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 2.1 MB/s | 141 kB 00:00 2023-11-24T06:24:59.531 INFO:teuthology.orchestra.run.smithi162.stdout:(4/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 6.8 MB/s | 2.1 MB 00:00 2023-11-24T06:24:59.631 INFO:teuthology.orchestra.run.smithi162.stdout:(5/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 4.6 MB/s | 5.2 MB 00:01 2023-11-24T06:24:59.806 INFO:teuthology.orchestra.run.smithi162.stdout:(6/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 5.3 MB/s | 1.5 MB 00:00 2023-11-24T06:25:00.223 INFO:teuthology.orchestra.run.smithi162.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 7.4 MB/s | 4.4 MB 00:00 2023-11-24T06:25:00.941 INFO:teuthology.orchestra.run.smithi162.stdout:(8/147): ceph-common-18.0.0-7484.ga13ea5ff.el9. 7.8 MB/s | 19 MB 00:02 2023-11-24T06:25:01.057 INFO:teuthology.orchestra.run.smithi162.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 208 kB/s | 24 kB 00:00 2023-11-24T06:25:01.224 INFO:teuthology.orchestra.run.smithi162.stdout:(10/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 8.6 MB/s | 8.6 MB 00:00 2023-11-24T06:25:01.324 INFO:teuthology.orchestra.run.smithi162.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.1 MB/s | 217 kB 00:00 2023-11-24T06:25:01.450 INFO:teuthology.orchestra.run.smithi162.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 251 kB/s | 31 kB 00:00 2023-11-24T06:25:01.489 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:25:01.491 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:25:01.514 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:25:01.515 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:25:01.517 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:25:01.518 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:25:01.521 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:25:01.525 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:25:01.527 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:25:01.567 INFO:teuthology.orchestra.run.smithi162.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 6.1 MB/s | 719 kB 00:00 2023-11-24T06:25:01.613 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-11-24T06:25:01.634 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:25:01.635 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:25:01.636 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:25:01.637 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:25:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:25:01.639 INFO:teuthology.orchestra.run.smithi064.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:25:01.640 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:25:01.641 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:25:01.642 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:25:01.643 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:25:01.644 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:25:01.645 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:25:01.647 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:25:01.648 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:25:01.649 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:25:01.650 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi064.stdout:Install 132 Packages 2023-11-24T06:25:01.652 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 15 Packages 2023-11-24T06:25:01.652 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:01.652 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 204 M 2023-11-24T06:25:01.653 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-11-24T06:25:01.658 INFO:teuthology.orchestra.run.smithi162.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.8 MB/s | 163 kB 00:00 2023-11-24T06:25:01.751 INFO:teuthology.orchestra.run.smithi162.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.4 MB/s | 126 kB 00:00 2023-11-24T06:25:02.043 INFO:teuthology.orchestra.run.smithi162.stdout:(16/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 7.9 MB/s | 18 MB 00:02 2023-11-24T06:25:02.085 INFO:teuthology.orchestra.run.smithi162.stdout:(17/147): libradosstriper1-18.0.0-7484.ga13ea5f 1.4 MB/s | 476 kB 00:00 2023-11-24T06:25:02.168 INFO:teuthology.orchestra.run.smithi162.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 546 kB/s | 45 kB 00:00 2023-11-24T06:25:02.285 INFO:teuthology.orchestra.run.smithi162.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 1.1 MB/s | 129 kB 00:00 2023-11-24T06:25:02.378 INFO:teuthology.orchestra.run.smithi162.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.7 MB/s | 162 kB 00:00 2023-11-24T06:25:02.486 INFO:teuthology.orchestra.run.smithi162.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 2.9 MB/s | 319 kB 00:00 2023-11-24T06:25:02.564 INFO:teuthology.orchestra.run.smithi064.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 22 kB/s | 6.5 kB 00:00 2023-11-24T06:25:02.577 INFO:teuthology.orchestra.run.smithi162.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 3.2 MB/s | 298 kB 00:00 2023-11-24T06:25:02.678 INFO:teuthology.orchestra.run.smithi162.stdout:(23/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 7.5 MB/s | 4.8 MB 00:00 2023-11-24T06:25:02.712 INFO:teuthology.orchestra.run.smithi162.stdout:(24/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 748 kB/s | 99 kB 00:00 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi162.stdout:(25/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.3 MB/s | 85 kB 00:00 2023-11-24T06:25:02.854 INFO:teuthology.orchestra.run.smithi162.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 1.5 MB/s | 170 kB 00:00 2023-11-24T06:25:02.987 INFO:teuthology.orchestra.run.smithi162.stdout:(27/147): ceph-grafana-dashboards-18.0.0-7484.g 212 kB/s | 28 kB 00:00 2023-11-24T06:25:03.104 INFO:teuthology.orchestra.run.smithi162.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.1 MB/s | 133 kB 00:00 2023-11-24T06:25:03.213 INFO:teuthology.orchestra.run.smithi162.stdout:(29/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 6.0 MB/s | 3.0 MB 00:00 2023-11-24T06:25:03.281 INFO:teuthology.orchestra.run.smithi064.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.2 MB/s | 859 kB 00:00 2023-11-24T06:25:03.439 INFO:teuthology.orchestra.run.smithi064.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 896 kB/s | 141 kB 00:00 2023-11-24T06:25:03.472 INFO:teuthology.orchestra.run.smithi162.stdout:(30/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 4.8 MB/s | 1.8 MB 00:00 2023-11-24T06:25:03.589 INFO:teuthology.orchestra.run.smithi162.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 2.1 MB/s | 243 kB 00:00 2023-11-24T06:25:03.714 INFO:teuthology.orchestra.run.smithi162.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 405 kB/s | 50 kB 00:00 2023-11-24T06:25:03.822 INFO:teuthology.orchestra.run.smithi162.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7484.ga 141 kB/s | 15 kB 00:00 2023-11-24T06:25:03.840 INFO:teuthology.orchestra.run.smithi064.stdout:(4/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 3.3 MB/s | 5.2 MB 00:01 2023-11-24T06:25:03.914 INFO:teuthology.orchestra.run.smithi162.stdout:(34/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 2.8 MB/s | 262 kB 00:00 2023-11-24T06:25:04.106 INFO:teuthology.orchestra.run.smithi162.stdout:(35/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 4.0 MB/s | 772 kB 00:00 2023-11-24T06:25:04.223 INFO:teuthology.orchestra.run.smithi162.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 157 kB/s | 18 kB 00:00 2023-11-24T06:25:04.265 INFO:teuthology.orchestra.run.smithi064.stdout:(5/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 2.5 MB/s | 2.1 MB 00:00 2023-11-24T06:25:04.315 INFO:teuthology.orchestra.run.smithi064.stdout:(6/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 3.1 MB/s | 1.5 MB 00:00 2023-11-24T06:25:04.416 INFO:teuthology.orchestra.run.smithi162.stdout:(37/147): ceph-mgr-diskprediction-local-18.0.0- 6.2 MB/s | 7.4 MB 00:01 2023-11-24T06:25:04.591 INFO:teuthology.orchestra.run.smithi162.stdout:(38/147): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2023-11-24T06:25:04.641 INFO:teuthology.orchestra.run.smithi162.stdout:(39/147): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 920 kB 00:00 2023-11-24T06:25:04.666 INFO:teuthology.orchestra.run.smithi162.stdout:(40/147): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2023-11-24T06:25:04.708 INFO:teuthology.orchestra.run.smithi162.stdout:(41/147): elfutils-libelf-devel-0.190-1.el9.x86 330 kB/s | 21 kB 00:00 2023-11-24T06:25:04.742 INFO:teuthology.orchestra.run.smithi162.stdout:(42/147): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2023-11-24T06:25:04.784 INFO:teuthology.orchestra.run.smithi162.stdout:(43/147): flexiblas-3.0.4-8.el9.x86_64.rpm 442 kB/s | 32 kB 00:00 2023-11-24T06:25:04.884 INFO:teuthology.orchestra.run.smithi162.stdout:(44/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2023-11-24T06:25:04.926 INFO:teuthology.orchestra.run.smithi162.stdout:(45/147): flexiblas-openblas-openmp-3.0.4-8.el9 133 kB/s | 17 kB 00:00 2023-11-24T06:25:04.960 INFO:teuthology.orchestra.run.smithi162.stdout:(46/147): jq-1.6-16.el9.x86_64.rpm 2.5 MB/s | 186 kB 00:00 2023-11-24T06:25:04.990 INFO:teuthology.orchestra.run.smithi064.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 6.1 MB/s | 4.4 MB 00:00 2023-11-24T06:25:05.002 INFO:teuthology.orchestra.run.smithi162.stdout:(47/147): libaio-devel-0.3.111-13.el9.x86_64.rp 138 kB/s | 10 kB 00:00 2023-11-24T06:25:05.027 INFO:teuthology.orchestra.run.smithi162.stdout:(48/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2023-11-24T06:25:05.069 INFO:teuthology.orchestra.run.smithi162.stdout:(49/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 682 kB/s | 44 kB 00:00 2023-11-24T06:25:05.119 INFO:teuthology.orchestra.run.smithi162.stdout:(50/147): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2023-11-24T06:25:05.161 INFO:teuthology.orchestra.run.smithi162.stdout:(51/147): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2023-11-24T06:25:05.203 INFO:teuthology.orchestra.run.smithi162.stdout:(52/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 314 kB/s | 24 kB 00:00 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi162.stdout:(53/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 557 kB/s | 46 kB 00:00 2023-11-24T06:25:05.279 INFO:teuthology.orchestra.run.smithi162.stdout:(54/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2023-11-24T06:25:05.337 INFO:teuthology.orchestra.run.smithi162.stdout:(55/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2023-11-24T06:25:05.388 INFO:teuthology.orchestra.run.smithi162.stdout:(56/147): openblas-0.3.21-2.el9.x86_64.rpm 322 kB/s | 34 kB 00:00 2023-11-24T06:25:05.530 INFO:teuthology.orchestra.run.smithi162.stdout:(57/147): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2023-11-24T06:25:05.580 INFO:teuthology.orchestra.run.smithi162.stdout:(58/147): postgresql-13.11-1.el9.x86_64.rpm 8.5 MB/s | 1.5 MB 00:00 2023-11-24T06:25:05.631 INFO:teuthology.orchestra.run.smithi162.stdout:(59/147): postgresql-contrib-13.11-1.el9.x86_64 8.0 MB/s | 811 kB 00:00 2023-11-24T06:25:05.656 INFO:teuthology.orchestra.run.smithi162.stdout:(60/147): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2023-11-24T06:25:05.716 INFO:teuthology.orchestra.run.smithi064.stdout:(8/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el9 12 MB/s | 8.6 MB 00:00 2023-11-24T06:25:05.766 INFO:teuthology.orchestra.run.smithi064.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 493 kB/s | 24 kB 00:00 2023-11-24T06:25:05.814 INFO:teuthology.orchestra.run.smithi162.stdout:(61/147): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2023-11-24T06:25:05.948 INFO:teuthology.orchestra.run.smithi162.stdout:(62/147): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2023-11-24T06:25:05.974 INFO:teuthology.orchestra.run.smithi162.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 1.5 MB/s | 228 kB 00:00 2023-11-24T06:25:06.049 INFO:teuthology.orchestra.run.smithi162.stdout:(64/147): python3-jsonpatch-1.21-16.el9.noarch. 330 kB/s | 24 kB 00:00 2023-11-24T06:25:06.099 INFO:teuthology.orchestra.run.smithi162.stdout:(65/147): python3-jsonpointer-2.0-4.el9.noarch. 146 kB/s | 18 kB 00:00 2023-11-24T06:25:06.116 INFO:teuthology.orchestra.run.smithi064.stdout:(10/147): ceph-common-18.0.0-7484.ga13ea5ff.el9 4.9 MB/s | 19 MB 00:03 2023-11-24T06:25:06.149 INFO:teuthology.orchestra.run.smithi162.stdout:(66/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T06:25:06.199 INFO:teuthology.orchestra.run.smithi162.stdout:(67/147): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 32 kB 00:00 2023-11-24T06:25:06.208 INFO:teuthology.orchestra.run.smithi064.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.4 MB/s | 217 kB 00:00 2023-11-24T06:25:06.250 INFO:teuthology.orchestra.run.smithi162.stdout:(68/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.4 MB/s | 217 kB 00:00 2023-11-24T06:25:06.300 INFO:teuthology.orchestra.run.smithi064.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 342 kB/s | 31 kB 00:00 2023-11-24T06:25:06.392 INFO:teuthology.orchestra.run.smithi162.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-11-24T06:25:06.417 INFO:teuthology.orchestra.run.smithi162.stdout:(70/147): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-11-24T06:25:06.492 INFO:teuthology.orchestra.run.smithi162.stdout:(71/147): python3-prettytable-0.7.2-27.el9.noar 558 kB/s | 41 kB 00:00 2023-11-24T06:25:06.517 INFO:teuthology.orchestra.run.smithi162.stdout:(72/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2023-11-24T06:25:06.568 INFO:teuthology.orchestra.run.smithi162.stdout:(73/147): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 211 kB 00:00 2023-11-24T06:25:06.600 INFO:teuthology.orchestra.run.smithi064.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 2.3 MB/s | 719 kB 00:00 2023-11-24T06:25:06.618 INFO:teuthology.orchestra.run.smithi162.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 478 kB/s | 47 kB 00:00 2023-11-24T06:25:06.675 INFO:teuthology.orchestra.run.smithi064.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 2.2 MB/s | 163 kB 00:00 2023-11-24T06:25:06.701 INFO:teuthology.orchestra.run.smithi162.stdout:(75/147): python3-requests-oauthlib-1.3.0-12.el 520 kB/s | 43 kB 00:00 2023-11-24T06:25:06.759 INFO:teuthology.orchestra.run.smithi064.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.5 MB/s | 126 kB 00:00 2023-11-24T06:25:06.985 INFO:teuthology.orchestra.run.smithi162.stdout:(76/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2023-11-24T06:25:07.001 INFO:teuthology.orchestra.run.smithi064.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 1.9 MB/s | 476 kB 00:00 2023-11-24T06:25:07.035 INFO:teuthology.orchestra.run.smithi162.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 121 kB/s | 39 kB 00:00 2023-11-24T06:25:07.260 INFO:teuthology.orchestra.run.smithi064.stdout:(17/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 6.0 MB/s | 18 MB 00:02 2023-11-24T06:25:07.327 INFO:teuthology.orchestra.run.smithi064.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 690 kB/s | 45 kB 00:00 2023-11-24T06:25:07.393 INFO:teuthology.orchestra.run.smithi064.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 1.9 MB/s | 129 kB 00:00 2023-11-24T06:25:07.485 INFO:teuthology.orchestra.run.smithi064.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.7 MB/s | 162 kB 00:00 2023-11-24T06:25:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:(78/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 6.4 MB/s | 41 MB 00:06 2023-11-24T06:25:07.519 INFO:teuthology.orchestra.run.smithi162.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 621 kB/s | 300 kB 00:00 2023-11-24T06:25:07.569 INFO:teuthology.orchestra.run.smithi162.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 106 kB/s | 56 kB 00:00 2023-11-24T06:25:07.586 INFO:teuthology.orchestra.run.smithi064.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 3.1 MB/s | 319 kB 00:00 2023-11-24T06:25:07.594 INFO:teuthology.orchestra.run.smithi162.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 815 kB/s | 85 kB 00:00 2023-11-24T06:25:07.645 INFO:teuthology.orchestra.run.smithi162.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 853 kB/s | 63 kB 00:00 2023-11-24T06:25:07.686 INFO:teuthology.orchestra.run.smithi064.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 2.9 MB/s | 298 kB 00:00 2023-11-24T06:25:07.695 INFO:teuthology.orchestra.run.smithi162.stdout:(83/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 357 kB/s | 44 kB 00:00 2023-11-24T06:25:07.721 INFO:teuthology.orchestra.run.smithi162.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 634 kB/s | 79 kB 00:00 2023-11-24T06:25:07.754 INFO:teuthology.orchestra.run.smithi162.stdout:(85/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.4 MB/s | 195 kB 00:00 2023-11-24T06:25:07.769 INFO:teuthology.orchestra.run.smithi064.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.2 MB/s | 99 kB 00:00 2023-11-24T06:25:07.804 INFO:teuthology.orchestra.run.smithi162.stdout:(86/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2023-11-24T06:25:07.829 INFO:teuthology.orchestra.run.smithi162.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2023-11-24T06:25:07.845 INFO:teuthology.orchestra.run.smithi064.stdout:(24/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.1 MB/s | 85 kB 00:00 2023-11-24T06:25:07.880 INFO:teuthology.orchestra.run.smithi162.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.1 MB 00:00 2023-11-24T06:25:07.905 INFO:teuthology.orchestra.run.smithi162.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2023-11-24T06:25:07.930 INFO:teuthology.orchestra.run.smithi162.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2023-11-24T06:25:07.955 INFO:teuthology.orchestra.run.smithi162.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-11-24T06:25:07.989 INFO:teuthology.orchestra.run.smithi162.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-24T06:25:08.206 INFO:teuthology.orchestra.run.smithi162.stdout:(93/147): python3-devel-3.9.17-2.el9.x86_64.rpm 990 kB/s | 245 kB 00:00 2023-11-24T06:25:08.290 INFO:teuthology.orchestra.run.smithi162.stdout:(94/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.0 MB/s | 309 kB 00:00 2023-11-24T06:25:08.473 INFO:teuthology.orchestra.run.smithi162.stdout:(95/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 7.6 MB/s | 4.1 MB 00:00 2023-11-24T06:25:08.517 INFO:teuthology.orchestra.run.smithi162.stdout:(96/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 109 kB/s | 24 kB 00:00 2023-11-24T06:25:08.562 INFO:teuthology.orchestra.run.smithi064.stdout:(25/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 15 MB/s | 41 MB 00:02 2023-11-24T06:25:08.679 INFO:teuthology.orchestra.run.smithi064.stdout:(26/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 2.9 MB/s | 4.8 MB 00:01 2023-11-24T06:25:08.691 INFO:teuthology.orchestra.run.smithi162.stdout:(97/147): liboath-2.6.7-2.el9.x86_64.rpm 225 kB/s | 49 kB 00:00 2023-11-24T06:25:08.713 INFO:teuthology.orchestra.run.smithi064.stdout:(27/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 1.1 MB/s | 170 kB 00:00 2023-11-24T06:25:08.730 INFO:teuthology.orchestra.run.smithi064.stdout:(28/147): ceph-grafana-dashboards-18.0.0-7484.g 571 kB/s | 28 kB 00:00 2023-11-24T06:25:08.733 INFO:teuthology.orchestra.run.smithi162.stdout:(98/147): libunwind-1.6.2-1.el9.x86_64.rpm 312 kB/s | 67 kB 00:00 2023-11-24T06:25:08.763 INFO:teuthology.orchestra.run.smithi064.stdout:(29/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 2.6 MB/s | 133 kB 00:00 2023-11-24T06:25:08.846 INFO:teuthology.orchestra.run.smithi064.stdout:(30/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 3.0 MB/s | 3.0 MB 00:01 2023-11-24T06:25:08.998 INFO:teuthology.orchestra.run.smithi064.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 1.6 MB/s | 243 kB 00:00 2023-11-24T06:25:09.089 INFO:teuthology.orchestra.run.smithi064.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 554 kB/s | 50 kB 00:00 2023-11-24T06:25:09.189 INFO:teuthology.orchestra.run.smithi064.stdout:(33/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 3.8 MB/s | 1.8 MB 00:00 2023-11-24T06:25:09.215 INFO:teuthology.orchestra.run.smithi064.stdout:(34/147): ceph-prometheus-alerts-18.0.0-7484.ga 122 kB/s | 15 kB 00:00 2023-11-24T06:25:09.249 INFO:teuthology.orchestra.run.smithi162.stdout:(99/147): python3-asyncssh-2.13.2-3.el9.noarch. 1.0 MB/s | 548 kB 00:00 2023-11-24T06:25:09.365 INFO:teuthology.orchestra.run.smithi064.stdout:(35/147): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-11-24T06:25:09.398 INFO:teuthology.orchestra.run.smithi064.stdout:(36/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 1.2 MB/s | 262 kB 00:00 2023-11-24T06:25:09.424 INFO:teuthology.orchestra.run.smithi064.stdout:(37/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 316 kB/s | 18 kB 00:00 2023-11-24T06:25:09.458 INFO:teuthology.orchestra.run.smithi162.stdout:(100/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 209 kB/s | 43 kB 00:00 2023-11-24T06:25:09.466 INFO:teuthology.orchestra.run.smithi064.stdout:(38/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 3.0 MB/s | 772 kB 00:00 2023-11-24T06:25:09.617 INFO:teuthology.orchestra.run.smithi162.stdout:(101/147): libarrow-9.0.0-9.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2023-11-24T06:25:09.659 INFO:teuthology.orchestra.run.smithi162.stdout:(102/147): python3-cachetools-4.2.4-1.el9.noarc 161 kB/s | 32 kB 00:00 2023-11-24T06:25:09.758 INFO:teuthology.orchestra.run.smithi064.stdout:(39/147): boost-program-options-1.75.0-8.el9.x8 610 kB/s | 106 kB 00:00 2023-11-24T06:25:09.784 INFO:teuthology.orchestra.run.smithi162.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 90 kB/s | 15 kB 00:00 2023-11-24T06:25:09.824 INFO:teuthology.orchestra.run.smithi064.stdout:(40/147): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 920 kB 00:00 2023-11-24T06:25:09.884 INFO:teuthology.orchestra.run.smithi162.stdout:(104/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 702 kB/s | 837 kB 00:01 2023-11-24T06:25:09.900 INFO:teuthology.orchestra.run.smithi064.stdout:(41/147): automake-1.16.2-8.el9.noarch.rpm 1.3 MB/s | 662 kB 00:00 2023-11-24T06:25:09.925 INFO:teuthology.orchestra.run.smithi064.stdout:(42/147): elfutils-libelf-devel-0.190-1.el9.x86 133 kB/s | 21 kB 00:00 2023-11-24T06:25:09.959 INFO:teuthology.orchestra.run.smithi064.stdout:(43/147): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 307 kB 00:00 2023-11-24T06:25:09.984 INFO:teuthology.orchestra.run.smithi064.stdout:(44/147): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 32 kB 00:00 2023-11-24T06:25:10.117 INFO:teuthology.orchestra.run.smithi064.stdout:(45/147): flexiblas-openblas-openmp-3.0.4-8.el9 133 kB/s | 17 kB 00:00 2023-11-24T06:25:10.151 INFO:teuthology.orchestra.run.smithi064.stdout:(46/147): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 186 kB 00:00 2023-11-24T06:25:10.209 INFO:teuthology.orchestra.run.smithi162.stdout:(105/147): python3-google-auth-2.23.4-1.el9.noa 609 kB/s | 197 kB 00:00 2023-11-24T06:25:10.235 INFO:teuthology.orchestra.run.smithi064.stdout:(47/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2023-11-24T06:25:10.260 INFO:teuthology.orchestra.run.smithi064.stdout:(48/147): libaio-devel-0.3.111-13.el9.x86_64.rp 72 kB/s | 10 kB 00:00 2023-11-24T06:25:10.293 INFO:teuthology.orchestra.run.smithi064.stdout:(49/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2023-11-24T06:25:10.310 INFO:teuthology.orchestra.run.smithi162.stdout:(106/147): python3-cherrypy-18.6.1-2.el9.noarch 683 kB/s | 358 kB 00:00 2023-11-24T06:25:10.327 INFO:teuthology.orchestra.run.smithi064.stdout:(50/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 688 kB/s | 44 kB 00:00 2023-11-24T06:25:10.377 INFO:teuthology.orchestra.run.smithi162.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2023-11-24T06:25:10.377 INFO:teuthology.orchestra.run.smithi064.stdout:(51/147): librdkafka-1.6.1-102.el9.x86_64.rpm 5.7 MB/s | 661 kB 00:00 2023-11-24T06:25:10.411 INFO:teuthology.orchestra.run.smithi064.stdout:(52/147): libtool-2.4.6-45.el9.x86_64.rpm 5.0 MB/s | 578 kB 00:00 2023-11-24T06:25:10.443 INFO:teuthology.orchestra.run.smithi162.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2023-11-24T06:25:10.452 INFO:teuthology.orchestra.run.smithi064.stdout:(53/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 207 kB/s | 24 kB 00:00 2023-11-24T06:25:10.469 INFO:teuthology.orchestra.run.smithi162.stdout:(109/147): python3-cheroot-8.6.0-4.el9.noarch.r 212 kB/s | 172 kB 00:00 2023-11-24T06:25:10.478 INFO:teuthology.orchestra.run.smithi064.stdout:(54/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 474 kB/s | 46 kB 00:00 2023-11-24T06:25:10.511 INFO:teuthology.orchestra.run.smithi064.stdout:(55/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-11-24T06:25:10.536 INFO:teuthology.orchestra.run.smithi064.stdout:(56/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2023-11-24T06:25:10.552 INFO:teuthology.orchestra.run.smithi162.stdout:(110/147): python3-jaraco-collections-3.0.0-8.e 132 kB/s | 23 kB 00:00 2023-11-24T06:25:10.562 INFO:teuthology.orchestra.run.smithi064.stdout:(57/147): openblas-0.3.21-2.el9.x86_64.rpm 420 kB/s | 34 kB 00:00 2023-11-24T06:25:10.611 INFO:teuthology.orchestra.run.smithi162.stdout:(111/147): python3-jaraco-text-3.2.0-6.el9.noar 139 kB/s | 20 kB 00:00 2023-11-24T06:25:10.636 INFO:teuthology.orchestra.run.smithi162.stdout:(112/147): python3-jaraco-functools-3.5.0-2.el9 101 kB/s | 19 kB 00:00 2023-11-24T06:25:10.754 INFO:teuthology.orchestra.run.smithi162.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 45 kB/s | 9.0 kB 00:00 2023-11-24T06:25:10.804 INFO:teuthology.orchestra.run.smithi064.stdout:(58/147): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2023-11-24T06:25:10.853 INFO:teuthology.orchestra.run.smithi162.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 169 kB/s | 41 kB 00:00 2023-11-24T06:25:10.871 INFO:teuthology.orchestra.run.smithi064.stdout:(59/147): postgresql-13.11-1.el9.x86_64.rpm 5.1 MB/s | 1.5 MB 00:00 2023-11-24T06:25:10.913 INFO:teuthology.orchestra.run.smithi064.stdout:(60/147): postgresql-contrib-13.11-1.el9.x86_64 2.3 MB/s | 811 kB 00:00 2023-11-24T06:25:10.938 INFO:teuthology.orchestra.run.smithi064.stdout:(61/147): postgresql-private-libs-13.11-1.el9.x 1.0 MB/s | 134 kB 00:00 2023-11-24T06:25:10.962 INFO:teuthology.orchestra.run.smithi162.stdout:(115/147): python3-logutils-0.3.5-21.el9.noarch 223 kB/s | 46 kB 00:00 2023-11-24T06:25:11.112 INFO:teuthology.orchestra.run.smithi162.stdout:(116/147): python3-natsort-7.1.1-5.el9.noarch.r 386 kB/s | 58 kB 00:00 2023-11-24T06:25:11.163 INFO:teuthology.orchestra.run.smithi064.stdout:(62/147): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2023-11-24T06:25:11.214 INFO:teuthology.orchestra.run.smithi064.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 841 kB/s | 228 kB 00:00 2023-11-24T06:25:11.271 INFO:teuthology.orchestra.run.smithi162.stdout:(117/147): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2023-11-24T06:25:11.356 INFO:teuthology.orchestra.run.smithi064.stdout:(64/147): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2023-11-24T06:25:11.373 INFO:teuthology.orchestra.run.smithi064.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 124 kB/s | 24 kB 00:00 2023-11-24T06:25:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:(66/147): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 18 kB 00:00 2023-11-24T06:25:11.438 INFO:teuthology.orchestra.run.smithi162.stdout:(118/147): python3-portend-3.1.0-2.el9.noarch.r 99 kB/s | 16 kB 00:00 2023-11-24T06:25:11.439 INFO:teuthology.orchestra.run.smithi064.stdout:(67/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T06:25:11.463 INFO:teuthology.orchestra.run.smithi162.stdout:(119/147): python3-more-itertools-8.12.0-2.el9. 129 kB/s | 79 kB 00:00 2023-11-24T06:25:11.482 INFO:teuthology.orchestra.run.smithi064.stdout:(68/147): python3-markupsafe-1.1.1-12.el9.x86_6 350 kB/s | 32 kB 00:00 2023-11-24T06:25:11.522 INFO:teuthology.orchestra.run.smithi162.stdout:(120/147): python3-pecan-1.4.2-3.el9.noarch.rpm 667 kB/s | 272 kB 00:00 2023-11-24T06:25:11.598 INFO:teuthology.orchestra.run.smithi064.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2023-11-24T06:25:11.623 INFO:teuthology.orchestra.run.smithi064.stdout:(70/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-11-24T06:25:11.649 INFO:teuthology.orchestra.run.smithi064.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-11-24T06:25:11.672 INFO:teuthology.orchestra.run.smithi162.stdout:(121/147): python3-repoze-lru-0.7-16.el9.noarch 148 kB/s | 31 kB 00:00 2023-11-24T06:25:11.674 INFO:teuthology.orchestra.run.smithi064.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 870 kB/s | 41 kB 00:00 2023-11-24T06:25:11.725 INFO:teuthology.orchestra.run.smithi064.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2023-11-24T06:25:11.749 INFO:teuthology.orchestra.run.smithi064.stdout:(74/147): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 211 kB 00:00 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi064.stdout:(75/147): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 47 kB 00:00 2023-11-24T06:25:11.800 INFO:teuthology.orchestra.run.smithi064.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 598 kB/s | 43 kB 00:00 2023-11-24T06:25:11.823 INFO:teuthology.orchestra.run.smithi162.stdout:(122/147): python3-rsa-4.9-2.el9.noarch.rpm 396 kB/s | 59 kB 00:00 2023-11-24T06:25:11.848 INFO:teuthology.orchestra.run.smithi162.stdout:(123/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 220 kB/s | 90 kB 00:00 2023-11-24T06:25:11.956 INFO:teuthology.orchestra.run.smithi162.stdout:(124/147): python3-routes-2.5.1-5.el9.noarch.rp 435 kB/s | 188 kB 00:00 2023-11-24T06:25:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 209 kB/s | 39 kB 00:00 2023-11-24T06:25:12.043 INFO:teuthology.orchestra.run.smithi064.stdout:(78/147): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2023-11-24T06:25:12.057 INFO:teuthology.orchestra.run.smithi162.stdout:(125/147): python3-typing-extensions-4.4.0-2.el 245 kB/s | 51 kB 00:00 2023-11-24T06:25:12.107 INFO:teuthology.orchestra.run.smithi162.stdout:(126/147): python3-tempora-5.0.0-2.el9.noarch.r 126 kB/s | 36 kB 00:00 2023-11-24T06:25:12.251 INFO:teuthology.orchestra.run.smithi064.stdout:(79/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2023-11-24T06:25:12.276 INFO:teuthology.orchestra.run.smithi064.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 206 kB/s | 56 kB 00:00 2023-11-24T06:25:12.301 INFO:teuthology.orchestra.run.smithi064.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 346 kB/s | 85 kB 00:00 2023-11-24T06:25:12.326 INFO:teuthology.orchestra.run.smithi064.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2023-11-24T06:25:12.340 INFO:teuthology.orchestra.run.smithi162.stdout:(127/147): python3-websocket-client-1.2.3-2.el9 317 kB/s | 90 kB 00:00 2023-11-24T06:25:12.376 INFO:teuthology.orchestra.run.smithi064.stdout:(83/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 450 kB/s | 44 kB 00:00 2023-11-24T06:25:12.402 INFO:teuthology.orchestra.run.smithi064.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 799 kB/s | 79 kB 00:00 2023-11-24T06:25:12.435 INFO:teuthology.orchestra.run.smithi064.stdout:(85/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.4 MB/s | 195 kB 00:00 2023-11-24T06:25:12.486 INFO:teuthology.orchestra.run.smithi064.stdout:(86/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2023-11-24T06:25:12.511 INFO:teuthology.orchestra.run.smithi064.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 241 kB 00:00 2023-11-24T06:25:12.569 INFO:teuthology.orchestra.run.smithi064.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 9.0 MB/s | 1.1 MB 00:00 2023-11-24T06:25:12.595 INFO:teuthology.orchestra.run.smithi064.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2023-11-24T06:25:12.620 INFO:teuthology.orchestra.run.smithi064.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-11-24T06:25:12.645 INFO:teuthology.orchestra.run.smithi064.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-11-24T06:25:12.657 INFO:teuthology.orchestra.run.smithi162.stdout:(128/147): python3-webob-1.8.7-6.el9.noarch.rpm 328 kB/s | 230 kB 00:00 2023-11-24T06:25:12.679 INFO:teuthology.orchestra.run.smithi064.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-24T06:25:12.683 INFO:teuthology.orchestra.run.smithi162.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 58 kB/s | 20 kB 00:00 2023-11-24T06:25:12.716 INFO:teuthology.orchestra.run.smithi162.stdout:(130/147): python3-werkzeug-2.0.3-3.el9.1.noarc 702 kB/s | 427 kB 00:00 2023-11-24T06:25:12.829 INFO:teuthology.orchestra.run.smithi064.stdout:(93/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.0 MB/s | 309 kB 00:00 2023-11-24T06:25:12.950 INFO:teuthology.orchestra.run.smithi162.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 658 kB/s | 192 kB 00:00 2023-11-24T06:25:13.046 INFO:teuthology.orchestra.run.smithi064.stdout:(94/147): libarrow-9.0.0-9.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2023-11-24T06:25:13.088 INFO:teuthology.orchestra.run.smithi064.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 600 kB/s | 24 kB 00:00 2023-11-24T06:25:13.130 INFO:teuthology.orchestra.run.smithi064.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2023-11-24T06:25:13.172 INFO:teuthology.orchestra.run.smithi064.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2023-11-24T06:25:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:(132/147): librados2-18.0.0-7484.ga13ea5ff.el9. 6.6 MB/s | 3.3 MB 00:00 2023-11-24T06:25:13.239 INFO:teuthology.orchestra.run.smithi064.stdout:(98/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 12 MB/s | 837 kB 00:00 2023-11-24T06:25:13.267 INFO:teuthology.orchestra.run.smithi162.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-11-24T06:25:13.297 INFO:teuthology.orchestra.run.smithi064.stdout:(99/147): python3-asyncssh-2.13.2-3.el9.noarch. 9.3 MB/s | 548 kB 00:00 2023-11-24T06:25:13.339 INFO:teuthology.orchestra.run.smithi064.stdout:(100/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2023-11-24T06:25:13.376 INFO:teuthology.orchestra.run.smithi162.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 21 MB/s | 2.2 MB 00:00 2023-11-24T06:25:13.381 INFO:teuthology.orchestra.run.smithi064.stdout:(101/147): python3-cachetools-4.2.4-1.el9.noarc 785 kB/s | 32 kB 00:00 2023-11-24T06:25:13.423 INFO:teuthology.orchestra.run.smithi064.stdout:(102/147): python3-certifi-2021.10.8-2.el9.noar 365 kB/s | 15 kB 00:00 2023-11-24T06:25:13.457 INFO:teuthology.orchestra.run.smithi064.stdout:(103/147): python3-devel-3.9.17-2.el9.x86_64.rp 303 kB/s | 245 kB 00:00 2023-11-24T06:25:13.459 INFO:teuthology.orchestra.run.smithi162.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 21 MB/s | 1.8 MB 00:00 2023-11-24T06:25:13.490 INFO:teuthology.orchestra.run.smithi064.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2023-11-24T06:25:13.541 INFO:teuthology.orchestra.run.smithi064.stdout:(105/147): python3-google-auth-2.23.4-1.el9.noa 3.9 MB/s | 197 kB 00:00 2023-11-24T06:25:13.543 INFO:teuthology.orchestra.run.smithi162.stdout:(136/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 5.1 MB/s | 3.0 MB 00:00 2023-11-24T06:25:13.568 INFO:teuthology.orchestra.run.smithi162.stdout:(137/147): NetworkManager-team-1.45.6-1.el9.x86 389 kB/s | 42 kB 00:00 2023-11-24T06:25:13.582 INFO:teuthology.orchestra.run.smithi064.stdout:(106/147): python3-jaraco-8.2.1-3.el9.noarch.rp 262 kB/s | 11 kB 00:00 2023-11-24T06:25:13.603 INFO:teuthology.orchestra.run.smithi162.stdout:(138/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-11-24T06:25:13.627 INFO:teuthology.orchestra.run.smithi162.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-11-24T06:25:13.649 INFO:teuthology.orchestra.run.smithi064.stdout:(107/147): python3-jaraco-classes-3.2.1-5.el9.n 269 kB/s | 18 kB 00:00 2023-11-24T06:25:13.669 INFO:teuthology.orchestra.run.smithi162.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 557 kB/s | 36 kB 00:00 2023-11-24T06:25:13.683 INFO:teuthology.orchestra.run.smithi064.stdout:(108/147): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2023-11-24T06:25:13.703 INFO:teuthology.orchestra.run.smithi162.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2023-11-24T06:25:13.736 INFO:teuthology.orchestra.run.smithi162.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.9 MB/s | 255 kB 00:00 2023-11-24T06:25:13.750 INFO:teuthology.orchestra.run.smithi064.stdout:(109/147): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2023-11-24T06:25:13.753 INFO:teuthology.orchestra.run.smithi162.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 466 kB/s | 22 kB 00:00 2023-11-24T06:25:13.767 INFO:teuthology.orchestra.run.smithi064.stdout:(110/147): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2023-11-24T06:25:13.784 INFO:teuthology.orchestra.run.smithi064.stdout:(111/147): python3-jaraco-text-3.2.0-6.el9.noar 603 kB/s | 20 kB 00:00 2023-11-24T06:25:13.801 INFO:teuthology.orchestra.run.smithi064.stdout:(112/147): python3-jwt+crypto-2.4.0-1.el9.noarc 277 kB/s | 9.0 kB 00:00 2023-11-24T06:25:13.803 INFO:teuthology.orchestra.run.smithi162.stdout:(144/147): thrift-0.15.0-2.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2023-11-24T06:25:13.826 INFO:teuthology.orchestra.run.smithi064.stdout:(113/147): python3-jwt-2.4.0-1.el9.noarch.rpm 992 kB/s | 41 kB 00:00 2023-11-24T06:25:13.837 INFO:teuthology.orchestra.run.smithi162.stdout:(145/147): xfsdump-3.1.12-4.el9.x86_64.rpm 3.1 MB/s | 321 kB 00:00 2023-11-24T06:25:13.868 INFO:teuthology.orchestra.run.smithi064.stdout:(114/147): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2023-11-24T06:25:13.918 INFO:teuthology.orchestra.run.smithi064.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2023-11-24T06:25:13.944 INFO:teuthology.orchestra.run.smithi064.stdout:(116/147): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2023-11-24T06:25:13.969 INFO:teuthology.orchestra.run.smithi064.stdout:(117/147): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2023-11-24T06:25:14.002 INFO:teuthology.orchestra.run.smithi064.stdout:(118/147): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2023-11-24T06:25:14.021 INFO:teuthology.orchestra.run.smithi162.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 4.6 MB/s | 1.2 MB 00:00 2023-11-24T06:25:14.028 INFO:teuthology.orchestra.run.smithi064.stdout:(119/147): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2023-11-24T06:25:14.053 INFO:teuthology.orchestra.run.smithi064.stdout:(120/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2023-11-24T06:25:14.078 INFO:teuthology.orchestra.run.smithi064.stdout:(121/147): python3-repoze-lru-0.7-16.el9.noarch 622 kB/s | 31 kB 00:00 2023-11-24T06:25:14.088 INFO:teuthology.orchestra.run.smithi162.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 7.7 MB/s | 2.2 MB 00:00 2023-11-24T06:25:14.096 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:25:14.096 INFO:teuthology.orchestra.run.smithi162.stdout:Total 13 MB/s | 204 MB 00:15 2023-11-24T06:25:14.112 INFO:teuthology.orchestra.run.smithi064.stdout:(122/147): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2023-11-24T06:25:14.137 INFO:teuthology.orchestra.run.smithi064.stdout:(123/147): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2023-11-24T06:25:14.163 INFO:teuthology.orchestra.run.smithi064.stdout:(124/147): python3-tempora-5.0.0-2.el9.noarch.r 718 kB/s | 36 kB 00:00 2023-11-24T06:25:14.188 INFO:teuthology.orchestra.run.smithi064.stdout:(125/147): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2023-11-24T06:25:14.222 INFO:teuthology.orchestra.run.smithi064.stdout:(126/147): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2023-11-24T06:25:14.238 INFO:teuthology.orchestra.run.smithi064.stdout:(127/147): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2023-11-24T06:25:14.306 INFO:teuthology.orchestra.run.smithi064.stdout:(128/147): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2023-11-24T06:25:14.317 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2023-11-24T06:25:14.331 INFO:teuthology.orchestra.run.smithi064.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi064.stdout:(130/147): openssl-devel-3.0.7-25.el9.x86_64.rp 2.3 MB/s | 4.1 MB 00:01 2023-11-24T06:25:14.456 INFO:teuthology.orchestra.run.smithi064.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2023-11-24T06:25:14.532 INFO:teuthology.orchestra.run.smithi064.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2023-11-24T06:25:14.582 INFO:teuthology.orchestra.run.smithi064.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-11-24T06:25:14.597 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2023-11-24T06:25:14.597 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2023-11-24T06:25:14.683 INFO:teuthology.orchestra.run.smithi064.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 23 MB/s | 2.2 MB 00:00 2023-11-24T06:25:14.766 INFO:teuthology.orchestra.run.smithi064.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 22 MB/s | 1.8 MB 00:00 2023-11-24T06:25:14.808 INFO:teuthology.orchestra.run.smithi064.stdout:(136/147): NetworkManager-team-1.45.6-1.el9.x86 1.0 MB/s | 42 kB 00:00 2023-11-24T06:25:14.921 INFO:teuthology.orchestra.run.smithi064.stdout:(137/147): librados2-18.0.0-7484.ga13ea5ff.el9. 6.7 MB/s | 3.3 MB 00:00 2023-11-24T06:25:14.958 INFO:teuthology.orchestra.run.smithi064.stdout:(138/147): NetworkManager-tui-1.45.6-1.el9.x86_ 1.6 MB/s | 245 kB 00:00 2023-11-24T06:25:14.984 INFO:teuthology.orchestra.run.smithi064.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 69 kB 00:00 2023-11-24T06:25:15.000 INFO:teuthology.orchestra.run.smithi064.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 900 kB/s | 36 kB 00:00 2023-11-24T06:25:15.034 INFO:teuthology.orchestra.run.smithi064.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2023-11-24T06:25:15.068 INFO:teuthology.orchestra.run.smithi064.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.8 MB/s | 255 kB 00:00 2023-11-24T06:25:15.085 INFO:teuthology.orchestra.run.smithi064.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 468 kB/s | 22 kB 00:00 2023-11-24T06:25:15.118 INFO:teuthology.orchestra.run.smithi064.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2023-11-24T06:25:15.202 INFO:teuthology.orchestra.run.smithi064.stdout:(145/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 4.1 MB/s | 3.0 MB 00:00 2023-11-24T06:25:15.369 INFO:teuthology.orchestra.run.smithi064.stdout:(146/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 8.7 MB/s | 2.2 MB 00:00 2023-11-24T06:25:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:(147/147): openssl-3.0.7-25.el9.x86_64.rpm 3.7 MB/s | 1.2 MB 00:00 2023-11-24T06:25:15.424 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:25:15.425 INFO:teuthology.orchestra.run.smithi064.stdout:Total 15 MB/s | 204 MB 00:13 2023-11-24T06:25:15.650 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-11-24T06:25:15.947 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-11-24T06:25:15.947 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-11-24T06:25:16.856 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2023-11-24T06:25:16.857 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2023-11-24T06:25:18.145 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-11-24T06:25:18.146 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-11-24T06:25:23.456 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2023-11-24T06:25:23.628 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:25:23.664 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:25:23.740 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:25:23.801 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:25:23.839 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:25:23.951 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:25:24.362 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:25:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:25:24.509 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-11-24T06:25:24.562 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:25:24.588 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:25:24.682 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:25:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:25:24.718 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:25:24.737 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:25:24.797 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:25:24.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:25:24.858 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:25:24.886 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:25:24.981 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:25:25.081 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:25:25.144 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:25:25.169 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:25:25.316 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:25:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:25:25.408 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:25:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:25:25.550 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:25:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:25:25.743 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:25:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:25:25.905 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:25:25.917 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:25:25.944 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:25:26.005 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:25:26.015 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:25:26.198 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:25:26.245 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:25:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:25:26.399 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:25:26.453 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:25:26.634 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:25:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:25:26.667 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:25:26.675 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:25:26.759 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:25:26.777 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:25:26.848 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:25:26.888 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:25:26.952 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:25:27.024 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:25:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:25:27.127 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:25:27.175 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:25:27.213 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:25:27.297 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:25:27.311 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:25:27.345 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:25:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:25:27.391 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:25:27.493 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:25:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:25:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:25:27.661 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:25:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:25:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:25:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:25:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:25:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:25:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:25:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:25:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:25:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:25:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:25:28.560 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:25:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:25:28.615 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:25:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:25:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:25:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:25:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:25:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:25:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:25:28.868 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:25:28.921 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:25:28.964 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:25:28.988 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:25:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:25:29.174 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:25:29.237 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:25:29.276 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:25:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:25:29.452 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:25:29.502 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:25:29.542 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:25:29.620 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:25:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:25:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:25:29.924 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:25:29.976 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:25:29.980 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:25:30.020 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:25:30.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:25:30.134 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:25:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:25:30.192 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:25:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:25:30.239 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:25:30.259 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:25:30.291 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:25:30.322 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:25:30.444 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:25:30.496 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:25:30.534 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:25:30.564 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:25:30.585 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:25:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:25:30.736 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:25:30.764 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:25:30.828 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:25:30.889 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:25:30.914 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:25:30.951 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:25:30.955 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:25:30.995 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:25:31.065 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:25:31.121 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:25:31.122 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:25:31.178 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:25:31.180 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:25:31.203 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:25:31.241 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:25:31.277 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:25:31.312 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:25:31.354 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:25:31.375 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:25:31.413 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:25:31.420 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:25:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:25:31.478 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:25:31.524 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:25:31.591 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:25:31.637 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:25:31.652 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:25:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:25:31.693 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:25:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:25:31.728 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:25:31.782 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:25:31.822 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:25:31.901 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:25:31.950 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:25:31.964 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:25:32.004 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:25:32.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:25:32.101 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:25:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:25:32.202 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:25:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:25:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:25:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:25:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:25:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:25:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:25:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:25:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:25:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:25:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:25:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:25:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:25:33.227 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:25:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:25:33.261 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:25:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:25:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:25:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:25:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:25:33.532 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:25:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:25:33.557 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:25:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:25:33.651 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:25:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:25:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:25:33.939 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:25:33.997 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:25:34.474 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:25:34.612 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:25:35.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:25:35.096 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:25:35.139 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:25:35.173 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:25:35.437 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:25:35.475 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:25:35.572 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:25:36.155 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:25:36.255 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:25:36.293 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:25:36.324 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:25:36.366 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:25:36.409 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:25:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:25:36.558 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:25:36.775 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:25:36.813 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:36.933 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:36.968 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:25:37.023 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:25:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:25:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:37.624 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:37.651 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:25:38.177 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:25:38.263 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:25:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:25:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:25:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:25:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:25:38.802 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:25:38.837 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:38.958 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:38.993 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:25:39.051 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:25:39.566 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:25:39.598 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:25:40.153 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:40.182 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:40.809 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:40.853 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:41.004 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:41.004 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:25:41.004 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:25:41.004 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:25:41.045 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:43.099 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:43.309 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:25:43.309 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:25:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:43.701 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:53.882 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:53.882 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2023-11-24T06:25:53.882 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2023-11-24T06:25:53.882 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2023-11-24T06:25:53.883 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2023-11-24T06:25:53.883 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2023-11-24T06:25:53.883 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2023-11-24T06:25:53.883 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2023-11-24T06:25:53.883 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:25:54.125 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:54.175 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:54.175 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:54.175 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:25:54.175 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:25:55.921 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:56.004 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:56.004 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:56.004 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:25:56.004 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:56.004 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:56.004 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:25:56.246 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:56.319 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:56.319 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:56.319 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:25:56.319 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:56.319 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:56.319 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:25:56.682 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2023-11-24T06:25:56.683 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:56.887 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:56.888 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:56.888 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:25:56.888 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:56.888 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:56.888 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:25:56.897 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:56.949 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:56.949 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:56.949 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:25:56.949 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:57.008 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:25:57.607 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:25:57.634 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:25:57.671 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:25:57.764 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:25:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:25:58.112 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:25:58.148 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:59.261 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:25:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:25:59.599 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:25:59.633 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:25:59.705 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:25:59.706 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:59.706 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:25:59.706 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:25:59.706 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:25:59.706 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:25:59.767 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:25:59.783 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:59.806 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:25:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:59.859 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:59.860 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:25:59.860 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:59.860 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:25:59.883 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:25:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:26:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:26:00.762 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:26:00.795 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:26:00.825 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:26:00.827 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:26:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:26:00.909 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:00.909 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:26:00.909 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:26:00.910 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:26:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:00.935 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:26:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:26:00.965 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:26:01.050 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:26:01.050 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:01.050 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:26:01.050 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:01.129 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:26:01.212 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:26:01.213 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:01.213 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:26:01.213 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:26:01.213 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:26:01.213 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:01.301 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:26:01.342 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:26:01.572 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:26:01.648 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:26:01.648 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:01.648 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:26:01.648 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:26:01.648 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:26:01.648 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:02.532 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:26:02.558 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:26:02.707 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:26:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:26:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:26:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:26:03.017 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:03.017 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:26:03.017 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:26:03.017 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:26:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:26:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:26:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:26:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:26:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:26:04.305 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:26:04.305 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:04.306 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:26:04.306 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:26:04.306 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:26:04.308 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:04.364 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:26:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:26:04.469 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:04.469 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:26:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:04.547 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:26:04.661 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:26:04.662 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:04.662 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:26:04.662 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:26:04.662 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:26:04.662 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:26:05.126 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:26:05.126 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:26:05.126 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:26:05.126 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:26:05.127 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:26:05.127 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:05.827 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:26:05.927 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:26:05.959 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:26:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:26:06.335 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:26:06.364 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:26:06.398 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:26:06.438 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:26:06.474 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:26:06.600 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:26:06.667 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:26:06.709 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:26:07.618 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:26:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:26:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:26:09.077 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:26:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:26:09.240 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:26:09.304 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:26:09.449 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:26:09.503 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:26:09.519 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:26:09.553 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:26:09.557 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:26:09.595 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:26:09.595 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:26:09.613 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:26:09.652 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:26:09.770 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:26:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:26:09.875 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:26:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:26:09.908 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:26:09.941 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:26:10.037 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:26:10.066 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:26:10.068 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:26:10.107 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:26:10.107 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:26:10.108 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:26:10.140 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:26:10.180 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:26:10.217 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:26:10.359 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:26:10.426 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:26:10.427 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:26:10.454 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:26:10.484 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:26:10.567 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:26:10.603 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:26:10.638 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:26:10.695 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:26:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:26:11.083 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:26:11.083 INFO:teuthology.orchestra.run.smithi162.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:26:11.083 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:11.447 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:26:11.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:26:11.561 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:26:13.006 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:26:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:26:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:26:13.277 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:26:13.446 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:26:13.482 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:26:13.516 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:26:13.550 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:26:13.593 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:26:13.593 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:26:13.774 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:26:13.834 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:26:13.888 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:26:13.920 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:26:13.921 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:26:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:26:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:26:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:26:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:26:14.481 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:26:14.481 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:26:14.630 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:26:14.674 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:26:14.708 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:26:14.746 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:26:14.873 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:26:15.146 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:26:15.147 INFO:teuthology.orchestra.run.smithi064.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:26:15.147 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:26:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:26:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:26:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:26:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:26:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:26:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:26:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:26:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:26:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:26:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:26:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:26:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:26:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:26:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:26:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:26:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:26:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:26:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:26:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:26:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:26:17.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:26:17.887 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:26:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:26:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:26:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:26:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:26:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:26:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:26:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:17.900 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2023-11-24T06:26:18.500 DEBUG:teuthology.parallel:result is None 2023-11-24T06:26:19.588 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:26:19.588 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:26:19.588 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:26:19.588 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:26:19.588 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:26:19.589 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:26:19.590 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:26:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:26:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:26:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:26:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:26:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:26:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:26:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:26:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:26:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:26:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:26:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:26:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:26:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:26:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:26:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:26:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:26:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:26:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:26:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:26:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:26:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:26:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:26:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:26:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:26:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:26:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:26:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:26:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:26:21.801 INFO:teuthology.orchestra.run.smithi064.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.802 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.803 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.804 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.804 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.804 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.804 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.804 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.804 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:26:21.805 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.806 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.807 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:26:21.808 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:26:21.809 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:26:21.810 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:26:21.811 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:26:21.812 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:26:21.813 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:26:21.814 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:26:21.815 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:26:21.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:21.815 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-11-24T06:26:22.384 DEBUG:teuthology.parallel:result is None 2023-11-24T06:26:22.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:26:22.527 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:26:22.586 INFO:teuthology.orchestra.run.smithi064.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:26:22.586 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:26:22.586 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:26:22.587 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:26:22.797 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:26:22.853 INFO:teuthology.orchestra.run.smithi162.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:26:22.853 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:26:22.853 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:26:22.853 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T06:26:22.854 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:22.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:26:22.945 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:22.945 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:26:23.024 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T06:26:23.024 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:23.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:26:23.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:26:23.219 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:23.220 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:26:23.293 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:26:23.414 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T06:26:23.415 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:23.415 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:26:23.499 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:26:23.617 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:23.617 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:26:23.711 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:26:23.829 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T06:26:23.830 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:23.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:26:23.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:26:24.026 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:24.026 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:26:24.102 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:26:24.221 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T06:26:24.295 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T06:26:24.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2023-11-24T06:26:24.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2023-11-24T06:26:24.402 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T06:26:24.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:26:24.408 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:26:24.504 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T06:26:24.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:26:24.509 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:26:24.615 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T06:26:24.615 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T06:26:24.615 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T06:26:24.616 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:24.616 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:26:24.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:24.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:24.970 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:24.970 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:26:25.028 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:25.301 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:25.409 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T06:26:25.409 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T06:26:25.409 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}', 'email': None, 'first_in_suite': False, 'job_id': '7465558', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3305, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMBWw4LPwMQvxVPgW6Z5nJVeimE42KGyRzyRz7kIiXdPqs+1FRULDPryrnIqWH/SVfFlOo5bSnajqLLE+P57dUU=', 'smithi162.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWKYd2+qmve4xjdoRtS92UFJvCTKl8gtKVkGgl5I4mgJOj2juuxiBXVDTfPPdQh3E8RHK5Ef3PVr73pU1bGv8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'check-counter': {'counters': {'mgr': [{'min': 4, 'name': 'finisher-volumes.complete_latency.avgcount'}, {'expected_val': 0, 'name': 'finisher-volumes.queue_len'}]}}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestPerModuleFinsherThread']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_05:41:33', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T06:26:25.410 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:26:25.417 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:26:25.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:26:25.473 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:26:25.560 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:25.560 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:26:25.602 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:26:25.603 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2023-11-24T06:26:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:26:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:25.685 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-11-24T06:26:25.685 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:25.685 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:25.686 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 06:26:15.637811467 +0000 2023-11-24T06:26:25.686 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 06:23:39.266961460 +0000 2023-11-24T06:26:25.686 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 06:23:39.266961460 +0000 2023-11-24T06:26:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 06:23:39.266961460 +0000 2023-11-24T06:26:25.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:26:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T06:26:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T06:26:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000358438 s, 1.4 MB/s 2023-11-24T06:26:25.809 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:26:25.855 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2023-11-24T06:26:25.943 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 06:26:15.638811447 +0000 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 06:23:40.364939190 +0000 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 06:23:40.364939190 +0000 2023-11-24T06:26:25.944 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 06:23:40.364939190 +0000 2023-11-24T06:26:25.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:26:26.061 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T06:26:26.061 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T06:26:26.061 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000307767 s, 1.7 MB/s 2023-11-24T06:26:26.064 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:26:26.109 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2023-11-24T06:26:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:26:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:26.195 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-11-24T06:26:26.195 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:26.196 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:26.196 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 06:26:15.639811427 +0000 2023-11-24T06:26:26.196 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 06:23:41.438917407 +0000 2023-11-24T06:26:26.196 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 06:23:41.438917407 +0000 2023-11-24T06:26:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 06:23:41.438917407 +0000 2023-11-24T06:26:26.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:26:26.314 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T06:26:26.315 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T06:26:26.315 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000295505 s, 1.7 MB/s 2023-11-24T06:26:26.318 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:26:26.365 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2023-11-24T06:26:26.406 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 06:26:15.640811408 +0000 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 06:23:42.509895685 +0000 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 06:23:42.509895685 +0000 2023-11-24T06:26:26.407 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 06:23:42.509895685 +0000 2023-11-24T06:26:26.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:26:26.521 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T06:26:26.521 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T06:26:26.522 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000299212 s, 1.7 MB/s 2023-11-24T06:26:26.524 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:26:26.566 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:26.567 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:26.567 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:26:26.628 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:26:26.628 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2023-11-24T06:26:26.684 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:26:26.684 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:26.685 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-11-24T06:26:26.685 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:26.685 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:26.685 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-11-24 06:26:11.548821811 +0000 2023-11-24T06:26:26.685 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-11-24 06:23:18.128655898 +0000 2023-11-24T06:26:26.685 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-11-24 06:23:18.128655898 +0000 2023-11-24T06:26:26.685 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-11-24 06:23:18.128655898 +0000 2023-11-24T06:26:26.685 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:26:26.776 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-11-24T06:26:26.776 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-11-24T06:26:26.776 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000333796 s, 1.5 MB/s 2023-11-24T06:26:26.780 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:26:26.835 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2023-11-24T06:26:26.883 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:26:26.883 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:26.883 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-11-24T06:26:26.883 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:26.884 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:26.884 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-11-24 06:26:11.548821811 +0000 2023-11-24T06:26:26.884 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-11-24 06:23:19.195633759 +0000 2023-11-24T06:26:26.884 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-11-24 06:23:19.195633759 +0000 2023-11-24T06:26:26.884 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-11-24 06:23:19.195633759 +0000 2023-11-24T06:26:26.884 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:26:26.973 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-11-24T06:26:26.973 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-11-24T06:26:26.974 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000384388 s, 1.3 MB/s 2023-11-24T06:26:26.977 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:26:27.034 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2023-11-24T06:26:27.086 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:26:27.086 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:27.086 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-11-24T06:26:27.086 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:27.086 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:27.087 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-11-24 06:26:11.549821791 +0000 2023-11-24T06:26:27.087 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-11-24 06:23:20.237612140 +0000 2023-11-24T06:26:27.087 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-11-24 06:23:20.237612140 +0000 2023-11-24T06:26:27.087 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-11-24 06:23:20.237612140 +0000 2023-11-24T06:26:27.087 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:26:27.168 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-11-24T06:26:27.169 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-11-24T06:26:27.169 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000322669 s, 1.6 MB/s 2023-11-24T06:26:27.172 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:26:27.222 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 690 Links: 1 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-11-24 06:26:11.550821771 +0000 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-11-24 06:23:21.259590935 +0000 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-11-24 06:23:21.259590935 +0000 2023-11-24T06:26:27.266 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-11-24 06:23:21.259590935 +0000 2023-11-24T06:26:27.267 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:26:27.387 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-11-24T06:26:27.387 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-11-24T06:26:27.387 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000332261 s, 1.5 MB/s 2023-11-24T06:26:27.393 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:26:27.446 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:27.446 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:26:27.446 INFO:tasks.ceph:Generating config... 2023-11-24T06:26:27.447 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T06:26:27.448 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T06:26:27.448 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T06:26:27.448 INFO:tasks.ceph:[client] pid file = 2023-11-24T06:26:27.448 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T06:26:27.448 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T06:26:27.448 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T06:26:27.449 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T06:26:27.449 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T06:26:27.449 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T06:26:27.449 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T06:26:27.450 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T06:26:27.451 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T06:26:27.451 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T06:26:27.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T06:26:27.562 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T06:26:27.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T06:26:27.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:26:27.754 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.64'), ('mon.b', '172.21.15.162'), ('mon.c', '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]')] 2023-11-24T06:26:27.755 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.64,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T06:26:27.756 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:27.756 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:26:27.801 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.64 --add b 172.21.15.162 --addv c '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: generated fsid fb91be48-aa54-48b4-b765-8060df323172 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = pacific 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 0 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:fsid fb91be48-aa54-48b4-b765-8060df323172 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:last_changed 2023-11-24T06:26:27.935252+0000 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:created 2023-11-24T06:26:27.935252+0000 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:min_mon_release 16 (pacific) 2023-11-24T06:26:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:election_strategy: 1 2023-11-24T06:26:27.938 INFO:teuthology.orchestra.run.smithi064.stdout:0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2023-11-24T06:26:27.938 INFO:teuthology.orchestra.run.smithi064.stdout:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.b 2023-11-24T06:26:27.938 INFO:teuthology.orchestra.run.smithi064.stdout:2: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.c 2023-11-24T06:26:27.938 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T06:26:27.940 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:26:27.984 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fb91be48-aa54-48b4-b765-8060df323172... 2023-11-24T06:26:27.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:26:28.029 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:26:28.174 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2023-11-24T06:26:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2023-11-24T06:26:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2023-11-24T06:26:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2023-11-24T06:26:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = fb91be48-aa54-48b4-b765-8060df323172 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2023-11-24T06:26:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug trim objects = true 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2023-11-24T06:26:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore allocator = bitmap 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2023-11-24T06:26:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2023-11-24T06:26:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug finisher = 20 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-11-24T06:26:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:26:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2023-11-24T06:26:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds balancer = 20 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T06:26:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2023-11-24T06:26:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2023-11-24T06:26:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2023-11-24T06:26:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2023-11-24T06:26:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-11-24T06:26:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-11-24T06:26:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-11-24T06:26:28.188 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2023-11-24T06:26:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2023-11-24T06:26:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2023-11-24T06:26:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2023-11-24T06:26:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2023-11-24T06:26:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:26:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:26:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2023-11-24T06:26:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.64,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = fb91be48-aa54-48b4-b765-8060df323172 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2023-11-24T06:26:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2023-11-24T06:26:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2023-11-24T06:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: bdev async discard = True 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: bdev enable discard = True 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore allocator = bitmap 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 1/20 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 1/20 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 4/10 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:26:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:26:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2023-11-24T06:26:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2023-11-24T06:26:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.202 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2023-11-24T06:26:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-11-24T06:26:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2023-11-24T06:26:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2023-11-24T06:26:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2023-11-24T06:26:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: debug finisher = 20 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2023-11-24T06:26:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:26:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2023-11-24T06:26:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2023-11-24T06:26:28.207 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2023-11-24T06:26:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2023-11-24T06:26:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2023-11-24T06:26:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2023-11-24T06:26:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2023-11-24T06:26:28.220 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T06:26:28.220 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T06:26:28.350 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T06:26:28.350 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:28.350 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T06:26:28.401 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:28.401 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T06:26:28.497 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:26:28.498 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:28.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:26:28.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:26:28.629 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:26:28.629 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:28.720 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:26:28.721 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:28.721 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:26:28.721 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:26:28.873 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:26:28.873 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:28.971 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T06:26:28.971 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T06:26:28.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:26:29.146 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:26:29.153 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:26:29.348 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:26:29.359 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T06:26:29.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:26:29.551 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:26:29.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T06:26:29.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:26:29.848 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:26:29.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T06:26:29.953 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:26:30.149 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:26:30.159 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T06:26:30.258 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:26:30.440 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:26:30.449 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T06:26:30.551 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T06:26:30.551 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T06:26:30.693 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T06:26:30.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T06:26:30.899 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T06:26:30.975 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T06:26:31.108 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T06:26:31.166 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T06:26:31.290 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T06:26:31.344 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T06:26:31.345 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:26:31.345 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T06:26:31.453 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:31.453 INFO:tasks.ceph:role: osd.0 2023-11-24T06:26:31.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:26:31.453 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:26:31.560 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:31.560 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:31.560 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:31.560 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:31.561 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:31.561 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:31.561 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:31.561 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:31.561 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:31.561 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:32.083 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T06:26:32.091 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T06:26:32.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T06:26:32.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T06:26:32.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T06:26:32.518 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:32.518 INFO:tasks.ceph:role: osd.1 2023-11-24T06:26:32.518 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:26:32.518 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:26:32.625 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:32.626 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:32.626 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:32.626 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:32.626 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:33.119 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T06:26:33.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T06:26:33.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T06:26:33.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T06:26:33.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T06:26:33.499 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:33.500 INFO:tasks.ceph:role: osd.2 2023-11-24T06:26:33.500 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:26:33.500 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:26:33.605 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:33.606 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:33.607 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:33.607 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:33.607 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:34.118 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T06:26:34.124 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T06:26:34.125 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T06:26:34.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T06:26:34.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:34.557 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:34.558 INFO:tasks.ceph:role: osd.3 2023-11-24T06:26:34.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:26:34.558 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:26:34.665 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:34.665 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:34.665 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:34.665 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:34.666 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:34.666 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:34.666 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:34.666 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:34.666 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:34.666 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:35.147 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T06:26:35.155 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T06:26:35.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:35.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:35.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:35.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:35.567+0000 7f10b21ab500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T06:26:35.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:35.573+0000 7f10b21ab500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T06:26:35.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:35.573+0000 7f10b21ab500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:26:35.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:35.573+0000 7f10b21ab500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:26:35.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:35.573+0000 7f10b21ab500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:26:35.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:35.582+0000 7f10b21ab500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T06:26:38.773 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T06:26:38.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:39.029+0000 7fb33d218500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T06:26:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:39.030+0000 7fb33d218500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T06:26:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:39.030+0000 7fb33d218500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:26:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:39.030+0000 7fb33d218500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:26:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:39.030+0000 7fb33d218500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:26:39.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:39.032+0000 7fb33d218500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T06:26:41.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T06:26:42.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:42.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:42.217+0000 7f5bf81e2500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T06:26:42.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:42.218+0000 7f5bf81e2500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T06:26:42.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:42.218+0000 7f5bf81e2500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:26:42.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:42.218+0000 7f5bf81e2500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:26:42.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:42.218+0000 7f5bf81e2500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:26:42.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:42.220+0000 7f5bf81e2500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T06:26:45.412 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T06:26:45.519 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:45.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:45.661+0000 7f45ca015500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T06:26:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:45.662+0000 7f45ca015500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T06:26:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:45.662+0000 7f45ca015500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:26:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:45.662+0000 7f45ca015500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:26:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:45.662+0000 7f45ca015500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:26:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:26:45.664+0000 7f45ca015500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T06:26:48.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:48.966 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:49.071 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:49.071 INFO:tasks.ceph:role: osd.4 2023-11-24T06:26:49.071 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:26:49.072 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:26:49.174 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:49.174 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:49.175 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:49.175 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:49.175 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:49.175 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:49.175 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:49.175 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:49.176 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:49.176 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:49.694 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-11-24T06:26:49.700 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-11-24T06:26:49.700 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:49.891 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:49.996 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:50.079 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:50.079 INFO:tasks.ceph:role: osd.5 2023-11-24T06:26:50.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:26:50.079 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:50.166 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:50.646 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-11-24T06:26:50.651 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-11-24T06:26:50.651 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:50.784 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:50.878 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T06:26:50.960 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:50.960 INFO:tasks.ceph:role: osd.6 2023-11-24T06:26:50.960 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:26:50.960 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:51.046 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:51.558 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-11-24T06:26:51.565 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-11-24T06:26:51.565 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T06:26:51.755 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T06:26:51.853 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T06:26:51.944 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:51.945 INFO:tasks.ceph:role: osd.7 2023-11-24T06:26:51.945 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:26:51.945 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:52.040 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:52.041 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:52.041 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:52.496 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-11-24T06:26:52.503 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-11-24T06:26:52.503 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T06:26:52.644 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T06:26:52.734 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:52.880 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:52.878+0000 7fab16a10500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T06:26:52.881 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:52.878+0000 7fab16a10500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T06:26:52.881 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:52.878+0000 7fab16a10500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:26:52.881 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:52.878+0000 7fab16a10500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:26:52.881 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:52.878+0000 7fab16a10500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:26:52.885 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:52.883+0000 7fab16a10500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T06:26:55.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:55.420 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:55.813 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:55.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:56.058 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:56.054+0000 7ff0ad20e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T06:26:56.058 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:56.055+0000 7ff0ad20e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T06:26:56.059 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:56.055+0000 7ff0ad20e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:26:56.059 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:56.055+0000 7ff0ad20e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:26:56.059 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:56.055+0000 7ff0ad20e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:26:56.061 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:56.058+0000 7ff0ad20e500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T06:26:59.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:59.338 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:59.476 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:59.473+0000 7f1b4ee1f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T06:26:59.477 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:59.473+0000 7f1b4ee1f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T06:26:59.477 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:59.473+0000 7f1b4ee1f500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:26:59.477 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:59.474+0000 7f1b4ee1f500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:26:59.477 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:59.474+0000 7f1b4ee1f500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:26:59.479 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:26:59.476+0000 7f1b4ee1f500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T06:27:02.405 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T06:27:02.504 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:27:02.646 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:02.642+0000 7f14a8e8c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T06:27:02.646 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:02.643+0000 7f14a8e8c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T06:27:02.646 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:02.643+0000 7f14a8e8c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:27:02.646 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:02.643+0000 7f14a8e8c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:27:02.647 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:02.643+0000 7f14a8e8c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:27:02.648 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:02.645+0000 7f14a8e8c500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T06:27:05.823 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T06:27:05.922 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T06:27:05.922 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:05.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T06:27:06.040 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:06.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T06:27:06.146 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:06.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T06:27:06.249 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:06.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T06:27:06.357 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:06.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T06:27:06.458 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:06.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T06:27:06.566 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:06.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T06:27:06.669 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:06.669 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T06:27:06.775 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:06.775 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T06:27:06.872 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:06.873 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T06:27:06.965 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:06.965 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T06:27:07.059 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:07.059 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T06:27:07.151 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:07.151 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T06:27:07.241 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:07.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T06:27:07.330 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:07.330 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T06:27:07.392 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:07.392 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T06:27:07.448 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:07.448 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T06:27:07.506 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:07.506 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T06:27:07.560 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T06:27:07.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:27:07.569 DEBUG:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:27:07.663 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2023-11-24T06:27:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAVQmBloVmkCBAA6iwoOA2UZF0lcB3gEpGU8g== 2023-11-24T06:27:07.663 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-11-24T06:27:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAVQmBl5LPBIBAAK8ZrBVR74BsWwdMl+o3pIQ== 2023-11-24T06:27:07.663 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAVQmBlyq59MhAArlK8MP3EbOGju5GCkdv2CA== 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAbQmBlTdDlIRAAWRp8wd1QtGOYfBwSPCGBYQ== 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAfQmBl4JnSARAAeFR52EVl6b4P/qr1q4KJhQ== 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAiQmBlhVMCDRAAlBHIaLz4BpzJRhgZ9XaCOQ== 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAlQmBlwjV4JxAALMfIK6BnDxZHx6/ZWfXnFw== 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAVQmBlFouwFBAAMw85ZoHBMGIUUyvoHbpwsA== 2023-11-24T06:27:07.664 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAWQmBlnzDUCBAATUyJ5k3zNicQ1LF6VcGuhg== 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAWQmBlEdAhGhAAGQy68F64p4yFOxLDRF/w8A== 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAsQmBlZ6tfNBAA0h5fM+S2ktgYd1V0Eeg94w== 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAwQmBlz1ZdAxAABR5oSb2a2Qx6JivcAXfndg== 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAzQmBl6V9OHBAAvRq6Ej/lF5GhuKVV73C3nA== 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2023-11-24T06:27:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA2QmBlG39mJhAAMooP5segdadKlTamXvsy2w== 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAWQmBlHzE7KRAAcsll2ZSOmVWCava8K2aOxA== 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout:[client.2] 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAWQmBlk5N8NRAA55v2xj4Lz9pKwiXNf3Jd8g== 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAXQmBla5RjBhAADHjknLo81m7W0rdy4F1K5w== 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout:[client.3] 2023-11-24T06:27:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAXQmBlcFw2ERAAm4XGpcaioQYWZBbFD3D1XA== 2023-11-24T06:27:07.667 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.y] 2023-11-24T06:27:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAVQmBloVmkCBAA6iwoOA2UZF0lcB3gEpGU8g== 2023-11-24T06:27:07.667 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2023-11-24T06:27:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAVQmBl5LPBIBAAK8ZrBVR74BsWwdMl+o3pIQ== 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAVQmBlyq59MhAArlK8MP3EbOGju5GCkdv2CA== 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAbQmBlTdDlIRAAWRp8wd1QtGOYfBwSPCGBYQ== 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAfQmBl4JnSARAAeFR52EVl6b4P/qr1q4KJhQ== 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAiQmBlhVMCDRAAlBHIaLz4BpzJRhgZ9XaCOQ== 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAlQmBlwjV4JxAALMfIK6BnDxZHx6/ZWfXnFw== 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2023-11-24T06:27:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAVQmBlFouwFBAAMw85ZoHBMGIUUyvoHbpwsA== 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAWQmBlnzDUCBAATUyJ5k3zNicQ1LF6VcGuhg== 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAWQmBlEdAhGhAAGQy68F64p4yFOxLDRF/w8A== 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAsQmBlZ6tfNBAA0h5fM+S2ktgYd1V0Eeg94w== 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAwQmBlz1ZdAxAABR5oSb2a2Qx6JivcAXfndg== 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.6] 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAzQmBl6V9OHBAAvRq6Ej/lF5GhuKVV73C3nA== 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.7] 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQA2QmBlG39mJhAAMooP5segdadKlTamXvsy2w== 2023-11-24T06:27:07.669 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2023-11-24T06:27:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAWQmBlHzE7KRAAcsll2ZSOmVWCava8K2aOxA== 2023-11-24T06:27:07.670 INFO:teuthology.orchestra.run.smithi162.stdout:[client.2] 2023-11-24T06:27:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAWQmBlk5N8NRAA55v2xj4Lz9pKwiXNf3Jd8g== 2023-11-24T06:27:07.670 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2023-11-24T06:27:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAXQmBla5RjBhAADHjknLo81m7W0rdy4F1K5w== 2023-11-24T06:27:07.670 INFO:teuthology.orchestra.run.smithi162.stdout:[client.3] 2023-11-24T06:27:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAXQmBlcFw2ERAAm4XGpcaioQYWZBbFD3D1XA== 2023-11-24T06:27:07.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:27:07.680 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:27:07.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:07.831 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:07.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:07.982 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:08.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.135 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.295 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.451 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.594 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.602 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:08.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:27:08.753 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:27:08.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:08.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:09.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:09.067 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:27:09.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.217 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.368 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.515 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.521 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.672 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:27:09.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:09.824 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:09.977 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:09.983 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:10.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:10.141 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:10.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:10.300 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:27:10.447 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T06:27:10.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T06:27:10.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:27:10.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T06:27:10.807 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T06:27:10.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:27:11.069 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T06:27:11.154 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T06:27:11.246 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:27:11.404 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T06:27:11.477 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:27:11.486 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:27:11.541 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T06:27:11.542 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T06:27:11.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T06:27:11.548 INFO:tasks.ceph.mon.a:Started 2023-11-24T06:27:11.548 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T06:27:11.548 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T06:27:11.585 INFO:tasks.ceph.mon.b:Started 2023-11-24T06:27:11.585 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T06:27:11.586 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T06:27:11.594 INFO:tasks.ceph.mon.c:Started 2023-11-24T06:27:11.595 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T06:27:11.595 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T06:27:11.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T06:27:11.601 INFO:tasks.ceph.mgr.y:Started 2023-11-24T06:27:11.601 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T06:27:11.601 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T06:27:11.608 INFO:tasks.ceph.mgr.x:Started 2023-11-24T06:27:11.608 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T06:27:11.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T06:27:11.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.792+0000 7f53130cf640 1 Processor -- start 2023-11-24T06:27:11.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.793+0000 7f53130cf640 1 -- start start 2023-11-24T06:27:11.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f53130cf640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f53130cf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f53130cf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c15c5f0 con 0x7f530c13df80 2023-11-24T06:27:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f53130cf640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c06c470 con 0x7f530c13e360 2023-11-24T06:27:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f53130cf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c06c750 con 0x7f530c15f9c0 2023-11-24T06:27:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f5310e6b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f530c13df80 0x7f530c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52496/0 (socket says 172.21.15.64:52496) 2023-11-24T06:27:11.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 learned_addr learned my addr 172.21.15.64:0/3451280467 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.794+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:11.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:27:11.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:27:11.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:11.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:11.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.795+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.996+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.996+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:12.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:12.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:11.997+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.398+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.398+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:27:12.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:12.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:12.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:12.399+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:27:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:27:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:27:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.201+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:13.202+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:27:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 msgr2=0x7f530c0cf7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c15f9c0 0x7f530c0cf7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f530c0cf290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f530c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> v1:172.21.15.64:6789/0 conn(0x7f530c13df80 legacy=0x7f530c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c06ca30 con 0x7f52f0006170 2023-11-24T06:27:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.794+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c06c470 con 0x7f52f00036a0 2023-11-24T06:27:14.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.795+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.796+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:14.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.796+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:14.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.796+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:14.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.796+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:27:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.997+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.998+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.998+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.998+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:14.998+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:27:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:15.399+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:15.400+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:15.400+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:15.400+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:15.400+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:27:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:16.201+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:16.202+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:16.202+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:16.202+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:16.202+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> v1:172.21.15.162:6789/0 conn(0x7f52f0006170 legacy=0x7f52f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 reap_dead start 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f000af10 con 0x7f52f000a580 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c06c470 con 0x7f52f00036a0 2023-11-24T06:27:17.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c06c750 con 0x7f530c13e360 2023-11-24T06:27:17.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:17.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:17.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:27:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.796+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:27:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.997+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.997+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:27:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:18.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:18.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:17.998+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:27:18.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.399+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.399+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.399+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.399+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.399+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.400+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.400+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.400+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.400+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:18.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:18.400+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:27:19.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.201+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:27:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:27:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:19.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f5310e6b640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:19.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:19.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:19.202+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:27:20.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 msgr2=0x7f52f000a000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:20.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.162:3300/0 conn(0x7f530c13e360 0x7f52f000a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 msgr2=0x7f52f0005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> v2:172.21.15.64:3300/0 conn(0x7f52f00036a0 0x7f52f0005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 >> v1:172.21.15.162:6789/0 conn(0x7f52f000a580 legacy=0x7f52f000f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f000ab40 con 0x7f52f0008d60 2023-11-24T06:27:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.795+0000 7f530a7fc640 1 -- 172.21.15.64:0/3451280467 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c06c470 con 0x7f52f00098e0 2023-11-24T06:27:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.796+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.797+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 msgr2=0x7f52f00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.797+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 msgr2=0x7f52f00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.797+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.797+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:27:21.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.998+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:21.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.999+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 msgr2=0x7f52f00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:21.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.999+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 msgr2=0x7f52f00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:21.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.999+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:21.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:20.999+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:27:21.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:21.400+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:21.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:21.401+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 msgr2=0x7f52f00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:27:21.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:21.401+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 msgr2=0x7f52f00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:27:21.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:21.401+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:27:21.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:21.401+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:27:22.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.040+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144025090 0 0) 0x7f52f000ab40 con 0x7f52f0008d60 2023-11-24T06:27:22.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.040+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52fc003520 con 0x7f52f0008d60 2023-11-24T06:27:22.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.042+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15492954 0 0) 0x7f52fc003520 con 0x7f52f0008d60 2023-11-24T06:27:22.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.042+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 msgr2=0x7f52f00086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:22.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.042+0000 7f530b7fe640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52f00098e0 0x7f52f00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.042+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00149a0 con 0x7f52f0008d60 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.042+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f5300001260 con 0x7f52f0008d60 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.042+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 >> v1:172.21.15.64:6789/0 conn(0x7f52f0008d60 legacy=0x7f52f00091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.043+0000 7f531166c640 1 -- 172.21.15.64:0/3451280467 reap_dead start 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.043+0000 7f530b7fe640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f52fc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.043+0000 7f530b7fe640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f52fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.043+0000 7f530b7fe640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f52fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.043+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c06c470 con 0x7f52fc0047c0 2023-11-24T06:27:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.043+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc00c150 con 0x7f52fc0039a0 2023-11-24T06:27:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.043+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc00c310 con 0x7f52fc0052a0 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.044+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f52fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.044+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f52fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f52fc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 msgr2=0x7f52fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f52fc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 msgr2=0x7f52fc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f52fc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f530bfff640 1 -- 172.21.15.64:0/3451280467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc003520 con 0x7f52fc0047c0 2023-11-24T06:27:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f531166c640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f52fc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:22.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.045+0000 7f5310e6b640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f52fc003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:22.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.046+0000 7f530bfff640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f52fc004c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009da0 tx=0x7f52fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c04f0c7e0a5b945 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:22.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.050+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8034400 con 0x7f52fc0047c0 2023-11-24T06:27:22.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.050+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f8030050 con 0x7f52fc0047c0 2023-11-24T06:27:22.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.050+0000 7f530b7fe640 1 -- 172.21.15.64:0/3451280467 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f80347c0 con 0x7f52fc0047c0 2023-11-24T06:27:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.051+0000 7f53130cf640 1 -- 172.21.15.64:0/3451280467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 msgr2=0x7f52fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.051+0000 7f53130cf640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f52fc004c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009da0 tx=0x7f52fc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.052+0000 7f53130cf640 1 -- 172.21.15.64:0/3451280467 shutdown_connections 2023-11-24T06:27:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.052+0000 7f53130cf640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f52fc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.052+0000 7f53130cf640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f52fc004c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.052+0000 7f53130cf640 1 --2- 172.21.15.64:0/3451280467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f52fc003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.052+0000 7f53130cf640 1 -- 172.21.15.64:0/3451280467 >> 172.21.15.64:0/3451280467 conn(0x7f530c06a800 msgr2=0x7f530c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.052+0000 7f53130cf640 1 -- 172.21.15.64:0/3451280467 shutdown_connections 2023-11-24T06:27:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.052+0000 7f53130cf640 1 -- 172.21.15.64:0/3451280467 wait complete. 2023-11-24T06:27:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.054+0000 7f53130cf640 1 Processor -- start 2023-11-24T06:27:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.055+0000 7f53130cf640 1 -- start start 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f53130cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f530c153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f53130cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f530c0ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f53130cf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f530c0abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f53130cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c153ad0 con 0x7f52fc0047c0 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f53130cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c13cdb0 con 0x7f52fc0039a0 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f53130cf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c13d090 con 0x7f52fc0052a0 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f530bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f530c0ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f530bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f530c0ab810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36268/0 (socket says 172.21.15.64:36268) 2023-11-24T06:27:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.056+0000 7f530bfff640 1 -- 172.21.15.64:0/1973279591 learned_addr learned my addr 172.21.15.64:0/1973279591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.057+0000 7f5310e6b640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f530c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.057+0000 7f531166c640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f530c0abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.057+0000 7f530bfff640 1 -- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 msgr2=0x7f530c0abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.057+0000 7f530bfff640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f530c0abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.057+0000 7f530bfff640 1 -- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 msgr2=0x7f530c153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.057+0000 7f530bfff640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f530c153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.057+0000 7f530bfff640 1 -- 172.21.15.64:0/1973279591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c13d370 con 0x7f52fc0047c0 2023-11-24T06:27:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.058+0000 7f530bfff640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f530c0ab810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f52f8037040 tx=0x7f52f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.058+0000 7f53097fa640 1 -- 172.21.15.64:0/1973279591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f803e470 con 0x7f52fc0047c0 2023-11-24T06:27:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.058+0000 7f53097fa640 1 -- 172.21.15.64:0/1973279591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f803e5d0 con 0x7f52fc0047c0 2023-11-24T06:27:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.058+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c13d650 con 0x7f52fc0047c0 2023-11-24T06:27:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.058+0000 7f53097fa640 1 -- 172.21.15.64:0/1973279591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f803e780 con 0x7f52fc0047c0 2023-11-24T06:27:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.059+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c13db70 con 0x7f52fc0047c0 2023-11-24T06:27:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.060+0000 7f53097fa640 1 -- 172.21.15.64:0/1973279591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f52f803b040 con 0x7f52fc0047c0 2023-11-24T06:27:22.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.060+0000 7f53097fa640 1 -- 172.21.15.64:0/1973279591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f52f8049af0 con 0x7f52fc0047c0 2023-11-24T06:27:22.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.060+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c0675d0 con 0x7f52fc0047c0 2023-11-24T06:27:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.064+0000 7f53097fa640 1 -- 172.21.15.64:0/1973279591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f52f804e060 con 0x7f52fc0047c0 2023-11-24T06:27:22.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.168+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f530c0b42a0 con 0x7f52fc0047c0 2023-11-24T06:27:22.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.173+0000 7f53097fa640 1 -- 172.21.15.64:0/1973279591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f52f8039070 con 0x7f52fc0047c0 2023-11-24T06:27:22.176 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2023-11-24T06:27:22.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.174+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 msgr2=0x7f530c0ab810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.174+0000 7f53130cf640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f530c0ab810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f52f8037040 tx=0x7f52f8004730 comp rx=0 tx=0).stop 2023-11-24T06:27:22.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.174+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 shutdown_connections 2023-11-24T06:27:22.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.174+0000 7f53130cf640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52fc0052a0 0x7f530c0abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.174+0000 7f53130cf640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52fc0047c0 0x7f530c0ab810 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.174+0000 7f53130cf640 1 --2- 172.21.15.64:0/1973279591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52fc0039a0 0x7f530c153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.174+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 >> 172.21.15.64:0/1973279591 conn(0x7f530c06a800 msgr2=0x7f530c15eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:22.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.175+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 shutdown_connections 2023-11-24T06:27:22.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:22.175+0000 7f53130cf640 1 -- 172.21.15.64:0/1973279591 wait complete. 2023-11-24T06:27:22.194 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T06:27:22.195 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T06:27:22.195 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:22.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T06:27:22.278 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:22.278 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T06:27:22.413 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:22.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T06:27:22.500 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:27:22.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T06:27:22.634 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:22.634 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T06:27:22.639 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:22.637+0000 7f43f5246040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:27:22.676 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:22.673+0000 7fd10de30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:27:22.701 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:22.698+0000 7f43f5246040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:27:22.725 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:22.725 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T06:27:22.739 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:22.737+0000 7fd10de30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:27:22.822 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:22.822 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T06:27:22.955 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:27:22.955 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T06:27:23.042 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 21fc9639-fe05-492e-8fcc-f24a92c35d8f 0 2023-11-24T06:27:23.278 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.276+0000 7f86429be640 1 Processor -- start 2023-11-24T06:27:23.279 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.276+0000 7f86429be640 1 -- start start 2023-11-24T06:27:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.278+0000 7f86429be640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f863c0d0910 0x7f863c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f86429be640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c06ea10 con 0x7f863c143940 2023-11-24T06:27:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f86429be640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c06ecf0 con 0x7f863c143d20 2023-11-24T06:27:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f86429be640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c06f3b0 con 0x7f863c0d0910 2023-11-24T06:27:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f863b7fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f863c143d20 0x7f863c0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57508/0 (socket says 172.21.15.162:57508) 2023-11-24T06:27:23.282 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f863b7fe640 1 -- 172.21.15.162:0/2838862320 learned_addr learned my addr 172.21.15.162:0/2838862320 (peer_addr_for_me v1:172.21.15.162:0/0) 2023-11-24T06:27:23.282 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f8640f5b640 1 --2- 172.21.15.162:0/2838862320 >> v2:172.21.15.64:3300/0 conn(0x7f863c0d0910 0x7f863c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.284 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376502716 0 0) 0x7f863c06ecf0 con 0x7f863c143d20 2023-11-24T06:27:23.284 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.279+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8624003520 con 0x7f863c143d20 2023-11-24T06:27:23.284 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397871422 0 0) 0x7f8624003520 con 0x7f863c143d20 2023-11-24T06:27:23.284 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 >> v2:172.21.15.64:3300/0 conn(0x7f863c0d0910 msgr2=0x7f863c06f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.284 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 --2- 172.21.15.162:0/2838862320 >> v2:172.21.15.64:3300/0 conn(0x7f863c0d0910 0x7f863c06f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.284 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 >> v1:172.21.15.64:6789/0 conn(0x7f863c143940 legacy=0x7f863c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.284 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c06efd0 con 0x7f863c143d20 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f8630001670 con 0x7f863c143d20 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 >> v1:172.21.15.162:6789/0 conn(0x7f863c143d20 legacy=0x7f863c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8624003970 0x7f8624003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86240047c0 0x7f8624004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8624005180 0x7f862400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c06f3b0 con 0x7f8624003970 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862400c120 con 0x7f86240047c0 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862400c2e0 con 0x7f8624005180 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f863b7fe640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86240047c0 0x7f8624004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.280+0000 7f8640f5b640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8624005180 0x7f862400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.285 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.281+0000 7f863bfff640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8624003970 0x7f8624003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.281+0000 7f863b7fe640 1 -- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8624005180 msgr2=0x7f862400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.281+0000 7f863b7fe640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8624005180 0x7f862400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.281+0000 7f863b7fe640 1 -- 172.21.15.162:0/2838862320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8624003970 msgr2=0x7f8624003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.281+0000 7f863b7fe640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8624003970 0x7f8624003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.281+0000 7f863b7fe640 1 -- 172.21.15.162:0/2838862320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8624003520 con 0x7f86240047c0 2023-11-24T06:27:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.281+0000 7f8640f5b640 1 -- 172.21.15.162:0/2838862320 reap_dead start 2023-11-24T06:27:23.286 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.282+0000 7f863b7fe640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86240047c0 0x7f8624004c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f863000c240 tx=0x7f86240057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5772ff3b22694b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.282+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630005ad0 con 0x7f86240047c0 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.282+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630005c50 con 0x7f86240047c0 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.282+0000 7f863affd640 1 -- 172.21.15.162:0/2838862320 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863002c830 con 0x7f86240047c0 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.283+0000 7f86429be640 1 -- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86240047c0 msgr2=0x7f8624004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.283+0000 7f86429be640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86240047c0 0x7f8624004c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f863000c240 tx=0x7f86240057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.283+0000 7f86429be640 1 -- 172.21.15.162:0/2838862320 shutdown_connections 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.283+0000 7f86429be640 1 --2- 172.21.15.162:0/2838862320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86240047c0 0x7f8624004c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.283+0000 7f86429be640 1 -- 172.21.15.162:0/2838862320 >> 172.21.15.162:0/2838862320 conn(0x7f863c0ccc70 msgr2=0x7f863c0cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:23.288 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.283+0000 7f86429be640 1 -- 172.21.15.162:0/2838862320 shutdown_connections 2023-11-24T06:27:23.289 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.283+0000 7f86429be640 1 -- 172.21.15.162:0/2838862320 wait complete. 2023-11-24T06:27:23.289 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.285+0000 7f86429be640 1 Processor -- start 2023-11-24T06:27:23.289 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.285+0000 7f86429be640 1 -- start start 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f86429be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86240047c0 0x7f863c13f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f86429be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c13fa30 0x7f863c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f86429be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c1e3530 0x7f863c1e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f86429be640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1e3ef0 con 0x7f863c1e3530 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f86429be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c13f770 con 0x7f863c13fa30 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f86429be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0b4940 con 0x7f86240047c0 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f863b7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c13fa30 0x7f863c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f863bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86240047c0 0x7f863c13f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f863b7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c13fa30 0x7f863c0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37236/0 (socket says 172.21.15.162:37236) 2023-11-24T06:27:23.291 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f863bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86240047c0 0x7f863c13f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:55010/0 (socket says 172.21.15.162:55010) 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.286+0000 7f863b7fe640 1 -- 172.21.15.162:0/3659577470 learned_addr learned my addr 172.21.15.162:0/3659577470 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f8640f5b640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c1e3530 0x7f863c1e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f863bfff640 1 -- 172.21.15.162:0/3659577470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c13fa30 msgr2=0x7f863c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f863bfff640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c13fa30 0x7f863c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f863bfff640 1 -- 172.21.15.162:0/3659577470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c1e3530 msgr2=0x7f863c1e39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f863bfff640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c1e3530 0x7f863c1e39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f863bfff640 1 -- 172.21.15.162:0/3659577470 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0b4c20 con 0x7f86240047c0 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f8640f5b640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c1e3530 0x7f863c1e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f863bfff640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86240047c0 0x7f863c13f230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f862c00e410 tx=0x7f862c00e8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f8638ff9640 1 -- 172.21.15.162:0/3659577470 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0052e0 con 0x7f86240047c0 2023-11-24T06:27:23.292 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f8638ff9640 1 -- 172.21.15.162:0/3659577470 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c005440 con 0x7f86240047c0 2023-11-24T06:27:23.293 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f8638ff9640 1 -- 172.21.15.162:0/3659577470 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c007270 con 0x7f86240047c0 2023-11-24T06:27:23.293 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.287+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c0b4f00 con 0x7f86240047c0 2023-11-24T06:27:23.293 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.288+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c0b54c0 con 0x7f86240047c0 2023-11-24T06:27:23.293 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.288+0000 7f8638ff9640 1 -- 172.21.15.162:0/3659577470 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f862c0067d0 con 0x7f86240047c0 2023-11-24T06:27:23.293 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.288+0000 7f8638ff9640 1 -- 172.21.15.162:0/3659577470 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f862c0039e0 con 0x7f86240047c0 2023-11-24T06:27:23.293 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.289+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c0b5650 con 0x7f86240047c0 2023-11-24T06:27:23.296 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.293+0000 7f8638ff9640 1 -- 172.21.15.162:0/3659577470 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f862c019070 con 0x7f86240047c0 2023-11-24T06:27:23.323 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.321+0000 7f43f5246040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:27:23.371 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.369+0000 7fd10de30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:27:23.374 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.372+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "21fc9639-fe05-492e-8fcc-f24a92c35d8f", "id": 0} v 0) v1 -- 0x7f863c0675d0 con 0x7f86240047c0 2023-11-24T06:27:23.382 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.380+0000 7f8638ff9640 1 -- 172.21.15.162:0/3659577470 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "21fc9639-fe05-492e-8fcc-f24a92c35d8f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f862c006cb0 con 0x7f86240047c0 2023-11-24T06:27:23.382 INFO:teuthology.orchestra.run.smithi162.stdout:0 2023-11-24T06:27:23.383 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86240047c0 msgr2=0x7f863c13f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.383 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86240047c0 0x7f863c13f230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f862c00e410 tx=0x7f862c00e8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 shutdown_connections 2023-11-24T06:27:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c1e3530 0x7f863c1e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c13fa30 0x7f863c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 --2- 172.21.15.162:0/3659577470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86240047c0 0x7f863c13f230 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 >> 172.21.15.162:0/3659577470 conn(0x7f863c0ccc70 msgr2=0x7f863c0ac990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 shutdown_connections 2023-11-24T06:27:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.381+0000 7f86429be640 1 -- 172.21.15.162:0/3659577470 wait complete. 2023-11-24T06:27:23.388 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.386+0000 7f43f5246040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:27:23.400 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 9ce466f0-b63b-4e7f-8dbc-50688e7358e5 1 2023-11-24T06:27:23.438 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.436+0000 7fd10de30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:27:23.442 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.439+0000 7f43f5246040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:27:23.493 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.490+0000 7fd10de30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:27:23.507 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.505+0000 7f43f5246040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:27:23.559 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.556+0000 7fd10de30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:27:23.559 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.556+0000 7f43f5246040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:27:23.583 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.581+0000 7f39568bb640 1 Processor -- start 2023-11-24T06:27:23.584 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.581+0000 7f39568bb640 1 -- start start 2023-11-24T06:27:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39568bb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f395015d4b0 0x7f39500d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39568bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39500cf9a0 0x7f39500cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39568bb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39500d0320 con 0x7f39500cddf0 2023-11-24T06:27:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39568bb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39500b2730 con 0x7f395015d4b0 2023-11-24T06:27:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39568bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500d1ec0 con 0x7f39500cf9a0 2023-11-24T06:27:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39558b9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f395015d4b0 0x7f39500d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39550b8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39500cf9a0 0x7f39500cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39558b9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f395015d4b0 0x7f39500d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37242/0 (socket says 172.21.15.162:37242) 2023-11-24T06:27:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39550b8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39500cf9a0 0x7f39500cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:55026/0 (socket says 172.21.15.162:55026) 2023-11-24T06:27:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39558b9640 1 -- 172.21.15.162:0/2384546475 learned_addr learned my addr 172.21.15.162:0/2384546475 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39558b9640 1 -- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39500cf9a0 msgr2=0x7f39500cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.583+0000 7f39558b9640 1 --2- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39500cf9a0 0x7f39500cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.587 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.584+0000 7f39558b9640 1 -- 172.21.15.162:0/2384546475 >> v1:172.21.15.64:6789/0 conn(0x7f39500cddf0 legacy=0x7f39500ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.587 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.584+0000 7f39558b9640 1 -- 172.21.15.162:0/2384546475 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500b29d0 con 0x7f395015d4b0 2023-11-24T06:27:23.587 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.584+0000 7f39550b8640 1 --2- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39500cf9a0 0x7f39500cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:23.587 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.585+0000 7f39558b9640 1 --2- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f395015d4b0 0x7f39500d1550 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f394c0098a0 tx=0x7f394c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c543a468a2cb3a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.587 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.585+0000 7f39548b7640 1 -- 172.21.15.162:0/2384546475 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c031640 con 0x7f395015d4b0 2023-11-24T06:27:23.587 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.585+0000 7f39548b7640 1 -- 172.21.15.162:0/2384546475 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c0317a0 con 0x7f395015d4b0 2023-11-24T06:27:23.588 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.585+0000 7f39548b7640 1 -- 172.21.15.162:0/2384546475 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c031b30 con 0x7f395015d4b0 2023-11-24T06:27:23.588 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 -- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f395015d4b0 msgr2=0x7f39500d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.588 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 --2- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f395015d4b0 0x7f39500d1550 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f394c0098a0 tx=0x7f394c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.588 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 -- 172.21.15.162:0/2384546475 shutdown_connections 2023-11-24T06:27:23.589 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 --2- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39500cf9a0 0x7f39500cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.589 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 --2- 172.21.15.162:0/2384546475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f395015d4b0 0x7f39500d1550 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.589 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 -- 172.21.15.162:0/2384546475 >> 172.21.15.162:0/2384546475 conn(0x7f39500d3000 msgr2=0x7f39500d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:23.589 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 -- 172.21.15.162:0/2384546475 shutdown_connections 2023-11-24T06:27:23.589 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.586+0000 7f39568bb640 1 -- 172.21.15.162:0/2384546475 wait complete. 2023-11-24T06:27:23.590 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.588+0000 7f39568bb640 1 Processor -- start 2023-11-24T06:27:23.590 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.588+0000 7f39568bb640 1 -- start start 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39568bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39500cddf0 0x7f395015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39568bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39500cf9a0 0x7f39500a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39568bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f395015d4b0 0x7f39500cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39568bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500ca560 con 0x7f39500cddf0 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39568bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500ca840 con 0x7f39500cf9a0 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39568bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500cab20 con 0x7f395015d4b0 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39500cf9a0 0x7f39500a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39560ba640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f395015d4b0 0x7f39500cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39500cf9a0 0x7f39500a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37254/0 (socket says 172.21.15.162:37254) 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 -- 172.21.15.162:0/3519734000 learned_addr learned my addr 172.21.15.162:0/3519734000 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39558b9640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39500cddf0 0x7f395015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 -- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f395015d4b0 msgr2=0x7f39500cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.592 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f395015d4b0 0x7f39500cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.593 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 -- 172.21.15.162:0/3519734000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39500cddf0 msgr2=0x7f395015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.593 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39500cddf0 0x7f395015a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.593 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39550b8640 1 -- 172.21.15.162:0/3519734000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940009400 con 0x7f39500cf9a0 2023-11-24T06:27:23.593 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.589+0000 7f39560ba640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f395015d4b0 0x7f39500cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:23.593 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39550b8640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39500cf9a0 0x7f39500a05f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f394000ebe0 tx=0x7f394000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39467fc640 1 -- 172.21.15.162:0/3519734000 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940009760 con 0x7f39500cf9a0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39467fc640 1 -- 172.21.15.162:0/3519734000 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000cc30 con 0x7f39500cf9a0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39467fc640 1 -- 172.21.15.162:0/3519734000 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000eee0 con 0x7f39500cf9a0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c0094d0 con 0x7f39500cf9a0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395015da10 con 0x7f39500cf9a0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39467fc640 1 -- 172.21.15.162:0/3519734000 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3940005c90 con 0x7f39500cf9a0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.590+0000 7f39467fc640 1 -- 172.21.15.162:0/3519734000 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f3940013930 con 0x7f39500cf9a0 2023-11-24T06:27:23.594 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.591+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3950067450 con 0x7f39500cf9a0 2023-11-24T06:27:23.596 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.594+0000 7f39467fc640 1 -- 172.21.15.162:0/3519734000 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f3940021060 con 0x7f39500cf9a0 2023-11-24T06:27:23.613 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.611+0000 7fd10de30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:27:23.674 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.672+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ce466f0-b63b-4e7f-8dbc-50688e7358e5", "id": 1} v 0) v1 -- 0x7f39500d0100 con 0x7f39500cf9a0 2023-11-24T06:27:23.679 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.676+0000 7f43f5246040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:27:23.682 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.680+0000 7f39467fc640 1 -- 172.21.15.162:0/3519734000 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ce466f0-b63b-4e7f-8dbc-50688e7358e5", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3940029020 con 0x7f39500cf9a0 2023-11-24T06:27:23.682 INFO:teuthology.orchestra.run.smithi162.stdout:1 2023-11-24T06:27:23.683 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39500cf9a0 msgr2=0x7f39500a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.683 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39500cf9a0 0x7f39500a05f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f394000ebe0 tx=0x7f394000c520 comp rx=0 tx=0).stop 2023-11-24T06:27:23.683 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 shutdown_connections 2023-11-24T06:27:23.683 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f395015d4b0 0x7f39500cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.684 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39500cf9a0 0x7f39500a05f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.684 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 --2- 172.21.15.162:0/3519734000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39500cddf0 0x7f395015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.684 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 >> 172.21.15.162:0/3519734000 conn(0x7f39500d3000 msgr2=0x7f39500ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:23.684 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 shutdown_connections 2023-11-24T06:27:23.684 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.681+0000 7f39568bb640 1 -- 172.21.15.162:0/3519734000 wait complete. 2023-11-24T06:27:23.699 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 30710438-2317-403e-bc18-09f2465b7000 2 2023-11-24T06:27:23.733 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.730+0000 7fd10de30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:27:23.763 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.761+0000 7f43f5246040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:27:23.818 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.816+0000 7fd10de30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:27:23.819 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.816+0000 7f43f5246040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:27:23.874 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.872+0000 7fd10de30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:27:23.881 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.879+0000 7f43f5246040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:27:23.929 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.926+0000 7f680e35c640 1 Processor -- start 2023-11-24T06:27:23.929 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.927+0000 7f680e35c640 1 -- start start 2023-11-24T06:27:23.931 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.928+0000 7f680e35c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6808160a20 0x7f68080cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.931 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680e35c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68080b3b20 0x7f68080cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.931 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680e35c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68080d1e00 con 0x7f6808160e00 2023-11-24T06:27:23.931 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680e35c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68080d3c40 con 0x7f6808160a20 2023-11-24T06:27:23.931 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680e35c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68080d3e50 con 0x7f68080b3b20 2023-11-24T06:27:23.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680db5b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68080b3b20 0x7f68080cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680db5b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68080b3b20 0x7f68080cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37262/0 (socket says 172.21.15.162:37262) 2023-11-24T06:27:23.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680db5b640 1 -- 172.21.15.162:0/117658068 learned_addr learned my addr 172.21.15.162:0/117658068 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:23.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680d35a640 1 --2- 172.21.15.162:0/117658068 >> v2:172.21.15.64:3300/0 conn(0x7f6808160a20 0x7f68080cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680db5b640 1 -- 172.21.15.162:0/117658068 >> v2:172.21.15.64:3300/0 conn(0x7f6808160a20 msgr2=0x7f68080cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680db5b640 1 --2- 172.21.15.162:0/117658068 >> v2:172.21.15.64:3300/0 conn(0x7f6808160a20 0x7f68080cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680db5b640 1 -- 172.21.15.162:0/117658068 >> v1:172.21.15.64:6789/0 conn(0x7f6808160e00 legacy=0x7f68080b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.933 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.929+0000 7f680db5b640 1 -- 172.21.15.162:0/117658068 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080d1a90 con 0x7f68080b3b20 2023-11-24T06:27:23.933 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.931+0000 7f680db5b640 1 --2- 172.21.15.162:0/117658068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68080b3b20 0x7f68080cfd90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f680808d550 tx=0x7f680002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65aaee0a5b399d6b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.933 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.931+0000 7f67fffff640 1 -- 172.21.15.162:0/117658068 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680002ca80 con 0x7f68080b3b20 2023-11-24T06:27:23.933 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.931+0000 7f67fffff640 1 -- 172.21.15.162:0/117658068 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680002cbe0 con 0x7f68080b3b20 2023-11-24T06:27:23.933 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.931+0000 7f67fffff640 1 -- 172.21.15.162:0/117658068 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000343f0 con 0x7f68080b3b20 2023-11-24T06:27:23.934 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 -- 172.21.15.162:0/117658068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68080b3b20 msgr2=0x7f68080cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 --2- 172.21.15.162:0/117658068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68080b3b20 0x7f68080cfd90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f680808d550 tx=0x7f680002bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 -- 172.21.15.162:0/117658068 shutdown_connections 2023-11-24T06:27:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 --2- 172.21.15.162:0/117658068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68080b3b20 0x7f68080cfd90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 --2- 172.21.15.162:0/117658068 >> v2:172.21.15.64:3300/0 conn(0x7f6808160a20 0x7f68080cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 -- 172.21.15.162:0/117658068 >> 172.21.15.162:0/117658068 conn(0x7f68080d3120 msgr2=0x7f68080d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 -- 172.21.15.162:0/117658068 shutdown_connections 2023-11-24T06:27:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.932+0000 7f680e35c640 1 -- 172.21.15.162:0/117658068 wait complete. 2023-11-24T06:27:23.937 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.935+0000 7f680e35c640 1 Processor -- start 2023-11-24T06:27:23.937 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.935+0000 7f680e35c640 1 -- start start 2023-11-24T06:27:23.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.936+0000 7f680e35c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68080b3b20 0x7f68080cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.936+0000 7f680e35c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6808160a20 0x7f680815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.936+0000 7f680e35c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6808160e00 0x7f680806c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.936+0000 7f680e35c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080cbb60 con 0x7f6808160e00 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.936+0000 7f680e35c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080caee0 con 0x7f6808160a20 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.936+0000 7f680e35c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68080cb1c0 con 0x7f68080b3b20 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68080b3b20 0x7f68080cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68080b3b20 0x7f68080cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:55042/0 (socket says 172.21.15.162:55042) 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 -- 172.21.15.162:0/1244546725 learned_addr learned my addr 172.21.15.162:0/1244546725 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680cb59640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6808160a20 0x7f680815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680db5b640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6808160e00 0x7f680806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 -- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6808160a20 msgr2=0x7f680815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6808160a20 0x7f680815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 -- 172.21.15.162:0/1244546725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6808160e00 msgr2=0x7f680806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6808160e00 0x7f680806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 -- 172.21.15.162:0/1244546725 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009b30 con 0x7f68080b3b20 2023-11-24T06:27:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.937+0000 7f680d35a640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68080b3b20 0x7f68080cbf80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f680400d5f0 tx=0x7f680400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.941 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:23.936+0000 7f43f5246040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:27:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.938+0000 7f67fdffb640 1 -- 172.21.15.162:0/1244546725 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040125f0 con 0x7f68080b3b20 2023-11-24T06:27:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.938+0000 7f67fdffb640 1 -- 172.21.15.162:0/1244546725 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804012750 con 0x7f68080b3b20 2023-11-24T06:27:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.938+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68080cb4a0 con 0x7f68080b3b20 2023-11-24T06:27:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.938+0000 7f67fdffb640 1 -- 172.21.15.162:0/1244546725 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804012b00 con 0x7f68080b3b20 2023-11-24T06:27:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.938+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680815bbf0 con 0x7f68080b3b20 2023-11-24T06:27:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.938+0000 7f67fdffb640 1 -- 172.21.15.162:0/1244546725 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f680401e460 con 0x7f68080b3b20 2023-11-24T06:27:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.938+0000 7f67fdffb640 1 -- 172.21.15.162:0/1244546725 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f680401eb30 con 0x7f68080b3b20 2023-11-24T06:27:23.943 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.939+0000 7fd10de30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:27:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.939+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6808067b60 con 0x7f68080b3b20 2023-11-24T06:27:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:23.941+0000 7f67fdffb640 1 -- 172.21.15.162:0/1244546725 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f680401ed40 con 0x7f68080b3b20 2023-11-24T06:27:23.998 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:23.996+0000 7fd10de30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:27:24.023 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.021+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "30710438-2317-403e-bc18-09f2465b7000", "id": 2} v 0) v1 -- 0x7f68080b4e00 con 0x7f68080b3b20 2023-11-24T06:27:24.031 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.028+0000 7f67fdffb640 1 -- 172.21.15.162:0/1244546725 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "30710438-2317-403e-bc18-09f2465b7000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f680401e7e0 con 0x7f68080b3b20 2023-11-24T06:27:24.031 INFO:teuthology.orchestra.run.smithi162.stdout:2 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68080b3b20 msgr2=0x7f68080cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68080b3b20 0x7f68080cbf80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f680400d5f0 tx=0x7f680400dab0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 shutdown_connections 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6808160e00 0x7f680806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6808160a20 0x7f680815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 --2- 172.21.15.162:0/1244546725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68080b3b20 0x7f68080cbf80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 >> 172.21.15.162:0/1244546725 conn(0x7f68080d3120 msgr2=0x7f68080b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 shutdown_connections 2023-11-24T06:27:24.033 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.030+0000 7f680e35c640 1 -- 172.21.15.162:0/1244546725 wait complete. 2023-11-24T06:27:24.055 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 05951407-33c9-404c-8593-d959d76b5632 3 2023-11-24T06:27:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.247+0000 7f664894e640 1 Processor -- start 2023-11-24T06:27:24.249 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.247+0000 7f664894e640 1 -- start start 2023-11-24T06:27:24.251 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.248+0000 7f664894e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f664415d4b0 0x7f66440d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.251 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.248+0000 7f664894e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f66440cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.251 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f664894e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440d0320 con 0x7f66440cddf0 2023-11-24T06:27:24.251 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f664894e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66440b2730 con 0x7f664415d4b0 2023-11-24T06:27:24.251 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f664894e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66440d1ec0 con 0x7f66440cf9a0 2023-11-24T06:27:24.251 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f66440cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.252 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f66437fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f664415d4b0 0x7f66440d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.252 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f66440cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:55046/0 (socket says 172.21.15.162:55046) 2023-11-24T06:27:24.252 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f66437fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f664415d4b0 0x7f66440d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37276/0 (socket says 172.21.15.162:37276) 2023-11-24T06:27:24.252 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6643fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f66440cddf0 0x7f66440ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57510/0 (socket says 172.21.15.162:57510) 2023-11-24T06:27:24.252 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6642ffd640 1 -- 172.21.15.162:0/2923348927 learned_addr learned my addr 172.21.15.162:0/2923348927 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6642ffd640 1 -- 172.21.15.162:0/2923348927 >> v2:172.21.15.162:3300/0 conn(0x7f664415d4b0 msgr2=0x7f66440d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6642ffd640 1 --2- 172.21.15.162:0/2923348927 >> v2:172.21.15.162:3300/0 conn(0x7f664415d4b0 0x7f66440d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6642ffd640 1 -- 172.21.15.162:0/2923348927 >> v1:172.21.15.162:6789/0 conn(0x7f66440cddf0 legacy=0x7f66440ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f6642ffd640 1 -- 172.21.15.162:0/2923348927 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440b29d0 con 0x7f66440cf9a0 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.249+0000 7f66437fe640 1 --2- 172.21.15.162:0/2923348927 >> v2:172.21.15.162:3300/0 conn(0x7f664415d4b0 0x7f66440d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.250+0000 7f6642ffd640 1 --2- 172.21.15.162:0/2923348927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f66440cfde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f66380098a0 tx=0x7f663802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a45c8d51df1443d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.251+0000 7f66427fc640 1 -- 172.21.15.162:0/2923348927 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638030840 con 0x7f66440cf9a0 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.251+0000 7f66427fc640 1 -- 172.21.15.162:0/2923348927 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66380309a0 con 0x7f66440cf9a0 2023-11-24T06:27:24.253 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.251+0000 7f66427fc640 1 -- 172.21.15.162:0/2923348927 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638030d30 con 0x7f66440cf9a0 2023-11-24T06:27:24.254 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.251+0000 7f664894e640 1 -- 172.21.15.162:0/2923348927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 msgr2=0x7f66440cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.254 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.251+0000 7f664894e640 1 --2- 172.21.15.162:0/2923348927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f66440cfde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f66380098a0 tx=0x7f663802b980 comp rx=0 tx=0).stop 2023-11-24T06:27:24.254 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.252+0000 7f664894e640 1 -- 172.21.15.162:0/2923348927 shutdown_connections 2023-11-24T06:27:24.255 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.252+0000 7f664894e640 1 --2- 172.21.15.162:0/2923348927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f66440cfde0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.255 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.252+0000 7f664894e640 1 --2- 172.21.15.162:0/2923348927 >> v2:172.21.15.162:3300/0 conn(0x7f664415d4b0 0x7f66440d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.255 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.252+0000 7f664894e640 1 -- 172.21.15.162:0/2923348927 >> 172.21.15.162:0/2923348927 conn(0x7f66440d3000 msgr2=0x7f66440d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:24.255 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.252+0000 7f664894e640 1 -- 172.21.15.162:0/2923348927 shutdown_connections 2023-11-24T06:27:24.255 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.252+0000 7f664894e640 1 -- 172.21.15.162:0/2923348927 wait complete. 2023-11-24T06:27:24.255 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.253+0000 7f664894e640 1 Processor -- start 2023-11-24T06:27:24.255 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.253+0000 7f664894e640 1 -- start start 2023-11-24T06:27:24.257 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.254+0000 7f664894e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66440cddf0 0x7f664415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.257 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.254+0000 7f664894e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f664415f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.254+0000 7f664894e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664415d4b0 0x7f66440cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.254+0000 7f664894e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664409d430 con 0x7f664415d4b0 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.254+0000 7f664894e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664415f760 con 0x7f66440cddf0 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.254+0000 7f664894e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664415fa40 con 0x7f66440cf9a0 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f66437fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66440cddf0 0x7f664415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f664415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f66437fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66440cddf0 0x7f664415ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37284/0 (socket says 172.21.15.162:37284) 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f66437fe640 1 -- 172.21.15.162:0/2782849247 learned_addr learned my addr 172.21.15.162:0/2782849247 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f6642ffd640 1 -- 172.21.15.162:0/2782849247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66440cddf0 msgr2=0x7f664415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f6642ffd640 1 --2- 172.21.15.162:0/2782849247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66440cddf0 0x7f664415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.258 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f6642ffd640 1 -- 172.21.15.162:0/2782849247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664415d4b0 msgr2=0x7f66440cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.259 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f6642ffd640 1 --2- 172.21.15.162:0/2782849247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664415d4b0 0x7f66440cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.259 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f6642ffd640 1 -- 172.21.15.162:0/2782849247 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634009400 con 0x7f66440cf9a0 2023-11-24T06:27:24.259 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.255+0000 7f6642ffd640 1 --2- 172.21.15.162:0/2782849247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f664415f130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6638033000 tx=0x7f6638004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.256+0000 7f662bfff640 1 -- 172.21.15.162:0/2782849247 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638039c10 con 0x7f66440cf9a0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.256+0000 7f662bfff640 1 -- 172.21.15.162:0/2782849247 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6638039db0 con 0x7f66440cf9a0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.256+0000 7f662bfff640 1 -- 172.21.15.162:0/2782849247 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638043640 con 0x7f66440cf9a0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.256+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66380094d0 con 0x7f66440cf9a0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.256+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664415da10 con 0x7f66440cf9a0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.256+0000 7f662bfff640 1 -- 172.21.15.162:0/2782849247 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6638043aa0 con 0x7f66440cf9a0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.256+0000 7f662bfff640 1 -- 172.21.15.162:0/2782849247 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f663803e070 con 0x7f66440cf9a0 2023-11-24T06:27:24.260 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.257+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66440cfe10 con 0x7f66440cf9a0 2023-11-24T06:27:24.261 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.260+0000 7f662bfff640 1 -- 172.21.15.162:0/2782849247 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6638034070 con 0x7f66440cf9a0 2023-11-24T06:27:24.340 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.338+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "05951407-33c9-404c-8593-d959d76b5632", "id": 3} v 0) v1 -- 0x7f6644067450 con 0x7f66440cf9a0 2023-11-24T06:27:24.348 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.346+0000 7f662bfff640 1 -- 172.21.15.162:0/2782849247 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "05951407-33c9-404c-8593-d959d76b5632", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f663804d460 con 0x7f66440cf9a0 2023-11-24T06:27:24.348 INFO:teuthology.orchestra.run.smithi162.stdout:3 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 msgr2=0x7f664415f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 --2- 172.21.15.162:0/2782849247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f664415f130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6638033000 tx=0x7f6638004730 comp rx=0 tx=0).stop 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 shutdown_connections 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 --2- 172.21.15.162:0/2782849247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664415d4b0 0x7f66440cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 --2- 172.21.15.162:0/2782849247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66440cf9a0 0x7f664415f130 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 --2- 172.21.15.162:0/2782849247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66440cddf0 0x7f664415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 >> 172.21.15.162:0/2782849247 conn(0x7f66440d3000 msgr2=0x7f664406c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 shutdown_connections 2023-11-24T06:27:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.347+0000 7f664894e640 1 -- 172.21.15.162:0/2782849247 wait complete. 2023-11-24T06:27:24.365 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new ea4ade87-f98b-48fc-8def-4bc9eaeef94a 4 2023-11-24T06:27:24.411 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:24.408+0000 7f43f5246040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:27:24.486 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:24.483+0000 7fd10de30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:27:24.540 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.538+0000 7fc59d9fa640 1 Processor -- start 2023-11-24T06:27:24.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.538+0000 7fc59d9fa640 1 -- start start 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d9fa640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc598160e00 0x7fc5980cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d9fa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5980cff60 0x7fc5980ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d9fa640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5980d1e00 con 0x7fc598160a20 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d9fa640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5980d3c40 con 0x7fc598160e00 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d9fa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5980d3e50 con 0x7fc5980cff60 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d1f9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5980cff60 0x7fc5980ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d1f9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5980cff60 0x7fc5980ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37294/0 (socket says 172.21.15.162:37294) 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d1f9640 1 -- 172.21.15.162:0/87217041 learned_addr learned my addr 172.21.15.162:0/87217041 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc597fff640 1 --2- 172.21.15.162:0/87217041 >> v2:172.21.15.64:3300/0 conn(0x7fc598160e00 0x7fc5980cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d1f9640 1 -- 172.21.15.162:0/87217041 >> v2:172.21.15.64:3300/0 conn(0x7fc598160e00 msgr2=0x7fc5980cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:24.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d1f9640 1 --2- 172.21.15.162:0/87217041 >> v2:172.21.15.64:3300/0 conn(0x7fc598160e00 0x7fc5980cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d1f9640 1 -- 172.21.15.162:0/87217041 >> v1:172.21.15.64:6789/0 conn(0x7fc598160a20 legacy=0x7fc5980cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:24.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.540+0000 7fc59d1f9640 1 -- 172.21.15.162:0/87217041 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5980d1a90 con 0x7fc5980cff60 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.541+0000 7fc59d1f9640 1 --2- 172.21.15.162:0/87217041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5980cff60 0x7fc5980ce270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc59808d550 tx=0x7fc59002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628d6419d719a934 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.542+0000 7fc5977fe640 1 -- 172.21.15.162:0/87217041 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59002ca80 con 0x7fc5980cff60 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.542+0000 7fc5977fe640 1 -- 172.21.15.162:0/87217041 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59002cbe0 con 0x7fc5980cff60 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.542+0000 7fc5977fe640 1 -- 172.21.15.162:0/87217041 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900343f0 con 0x7fc5980cff60 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.542+0000 7fc59d9fa640 1 -- 172.21.15.162:0/87217041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5980cff60 msgr2=0x7fc5980ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.542+0000 7fc59d9fa640 1 --2- 172.21.15.162:0/87217041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5980cff60 0x7fc5980ce270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc59808d550 tx=0x7fc59002bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.543+0000 7fc59d9fa640 1 -- 172.21.15.162:0/87217041 shutdown_connections 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.543+0000 7fc59d9fa640 1 --2- 172.21.15.162:0/87217041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5980cff60 0x7fc5980ce270 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.545 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.543+0000 7fc59d9fa640 1 --2- 172.21.15.162:0/87217041 >> v2:172.21.15.64:3300/0 conn(0x7fc598160e00 0x7fc5980cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.543+0000 7fc59d9fa640 1 -- 172.21.15.162:0/87217041 >> 172.21.15.162:0/87217041 conn(0x7fc5980d3120 msgr2=0x7fc5980d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:24.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.543+0000 7fc59d9fa640 1 -- 172.21.15.162:0/87217041 shutdown_connections 2023-11-24T06:27:24.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.543+0000 7fc59d9fa640 1 -- 172.21.15.162:0/87217041 wait complete. 2023-11-24T06:27:24.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.544+0000 7fc59d9fa640 1 Processor -- start 2023-11-24T06:27:24.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.544+0000 7fc59d9fa640 1 -- start start 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.545+0000 7fc59d9fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5980cff60 0x7fc59815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.545+0000 7fc59d9fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc598160a20 0x7fc59806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.545+0000 7fc59d9fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc598160e00 0x7fc59806c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.545+0000 7fc59d9fa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5980cbf20 con 0x7fc598160e00 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.545+0000 7fc59d9fa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59815cf40 con 0x7fc598160a20 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.545+0000 7fc59d9fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5980cb2e0 con 0x7fc5980cff60 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc598160a20 0x7fc59806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc598160a20 0x7fc59806bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37306/0 (socket says 172.21.15.162:37306) 2023-11-24T06:27:24.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 -- 172.21.15.162:0/1835007131 learned_addr learned my addr 172.21.15.162:0/1835007131 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:24.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc59c9f8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5980cff60 0x7fc59815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.550 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc59d1f9640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc598160e00 0x7fc59806c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.550 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 -- 172.21.15.162:0/1835007131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5980cff60 msgr2=0x7fc59815ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.550 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5980cff60 0x7fc59815ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.550 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 -- 172.21.15.162:0/1835007131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc598160e00 msgr2=0x7fc59806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.550 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc598160e00 0x7fc59806c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.550 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 -- 172.21.15.162:0/1835007131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590009b30 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.546+0000 7fc597fff640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc598160a20 0x7fc59806bfe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc58400d5f0 tx=0x7fc58400dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.547+0000 7fc5957fa640 1 -- 172.21.15.162:0/1835007131 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5840125f0 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.547+0000 7fc5957fa640 1 -- 172.21.15.162:0/1835007131 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc584012750 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.547+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5980cb580 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.547+0000 7fc5957fa640 1 -- 172.21.15.162:0/1835007131 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584012b00 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.547+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5980cbb40 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.547+0000 7fc5957fa640 1 -- 172.21.15.162:0/1835007131 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc58401e460 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.547+0000 7fc5957fa640 1 -- 172.21.15.162:0/1835007131 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fc5840149e0 con 0x7fc598160a20 2023-11-24T06:27:24.551 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.548+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc598067b60 con 0x7fc598160a20 2023-11-24T06:27:24.552 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.550+0000 7fc5957fa640 1 -- 172.21.15.162:0/1835007131 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc584014c30 con 0x7fc598160a20 2023-11-24T06:27:24.579 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:24.577+0000 7f43f5246040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:27:24.632 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.629+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea4ade87-f98b-48fc-8def-4bc9eaeef94a", "id": 4} v 0) v1 -- 0x7fc5980d14e0 con 0x7fc598160a20 2023-11-24T06:27:24.639 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.636+0000 7fc5957fa640 1 -- 172.21.15.162:0/1835007131 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea4ade87-f98b-48fc-8def-4bc9eaeef94a", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc5840144d0 con 0x7fc598160a20 2023-11-24T06:27:24.639 INFO:teuthology.orchestra.run.smithi162.stdout:4 2023-11-24T06:27:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc598160a20 msgr2=0x7fc59806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc598160a20 0x7fc59806bfe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc58400d5f0 tx=0x7fc58400dab0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 shutdown_connections 2023-11-24T06:27:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc598160e00 0x7fc59806c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc598160a20 0x7fc59806bfe0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 --2- 172.21.15.162:0/1835007131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5980cff60 0x7fc59815ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 >> 172.21.15.162:0/1835007131 conn(0x7fc5980d3120 msgr2=0x7fc5980b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 shutdown_connections 2023-11-24T06:27:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.638+0000 7fc59d9fa640 1 -- 172.21.15.162:0/1835007131 wait complete. 2023-11-24T06:27:24.656 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:24.653+0000 7fd10de30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:27:24.657 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c 5 2023-11-24T06:27:24.718 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:24.715+0000 7f43f5246040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:27:24.773 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:24.771+0000 7f43f5246040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:27:24.807 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:24.804+0000 7fd10de30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.833+0000 7f30757a9640 1 Processor -- start 2023-11-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.833+0000 7f30757a9640 1 -- start start 2023-11-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.835+0000 7f30757a9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f307009a170 0x7f30700b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.835+0000 7f30757a9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307006cea0 con 0x7f307006c6b0 2023-11-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.835+0000 7f30757a9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307006cbc0 con 0x7f30700cd240 2023-11-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.835+0000 7f30757a9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307006f350 con 0x7f307009a170 2023-11-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.835+0000 7f306effd640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f30700cd240 0x7f3070099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57514/0 (socket says 172.21.15.162:57514) 2023-11-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.835+0000 7f306effd640 1 -- 172.21.15.162:0/1539782054 learned_addr learned my addr 172.21.15.162:0/1539782054 (peer_addr_for_me v1:172.21.15.162:0/0) 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f3067fff640 1 --2- 172.21.15.162:0/1539782054 >> v2:172.21.15.64:3300/0 conn(0x7f307009a170 0x7f30700b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438512997 0 0) 0x7f307006cbc0 con 0x7f30700cd240 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f305c003520 con 0x7f30700cd240 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210336882 0 0) 0x7f305c003520 con 0x7f30700cd240 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 >> v2:172.21.15.64:3300/0 conn(0x7f307009a170 msgr2=0x7f30700b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 --2- 172.21.15.162:0/1539782054 >> v2:172.21.15.64:3300/0 conn(0x7f307009a170 0x7f30700b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 >> v1:172.21.15.64:6789/0 conn(0x7f307006c6b0 legacy=0x7f30700cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307006f630 con 0x7f30700cd240 2023-11-24T06:27:24.844 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f3060001670 con 0x7f30700cd240 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 >> v1:172.21.15.162:6789/0 conn(0x7f30700cd240 legacy=0x7f3070099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c003970 0x7f305c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c0047c0 0x7f305c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f305c005180 0x7f305c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307006f350 con 0x7f305c0047c0 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c00c0b0 con 0x7f305c003970 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.836+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c00c270 con 0x7f305c005180 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f3067fff640 1 --2- 172.21.15.162:0/1539782054 >> v2:172.21.15.64:3300/0 conn(0x7f307009a170 0x7f30700b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306effd640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c003970 0x7f305c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306f7fe640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f305c005180 0x7f305c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306effd640 1 -- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f305c005180 msgr2=0x7f305c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:24.845 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306effd640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f305c005180 0x7f305c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306effd640 1 -- 172.21.15.162:0/1539782054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c0047c0 msgr2=0x7f305c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f3067fff640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c0047c0 0x7f305c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306effd640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c0047c0 0x7f305c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306f7fe640 1 -- 172.21.15.162:0/1539782054 reap_dead start 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.837+0000 7f306effd640 1 -- 172.21.15.162:0/1539782054 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c003520 con 0x7f305c003970 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.838+0000 7f306effd640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c003970 0x7f305c003d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3060001eb0 tx=0x7f305c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90977ec951ae229b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.838+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060005ad0 con 0x7f305c003970 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.838+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060005c50 con 0x7f305c003970 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.838+0000 7f306e7fc640 1 -- 172.21.15.162:0/1539782054 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306002c850 con 0x7f305c003970 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.839+0000 7f30757a9640 1 -- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c003970 msgr2=0x7f305c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.846 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.839+0000 7f30757a9640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c003970 0x7f305c003d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3060001eb0 tx=0x7f305c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.847 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.839+0000 7f30757a9640 1 -- 172.21.15.162:0/1539782054 shutdown_connections 2023-11-24T06:27:24.847 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.839+0000 7f30757a9640 1 --2- 172.21.15.162:0/1539782054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c003970 0x7f305c003d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.847 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.839+0000 7f30757a9640 1 -- 172.21.15.162:0/1539782054 >> 172.21.15.162:0/1539782054 conn(0x7f30700b2d00 msgr2=0x7f30700b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:24.847 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.839+0000 7f30757a9640 1 -- 172.21.15.162:0/1539782054 shutdown_connections 2023-11-24T06:27:24.847 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.839+0000 7f30757a9640 1 -- 172.21.15.162:0/1539782054 wait complete. 2023-11-24T06:27:24.847 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.841+0000 7f30757a9640 1 Processor -- start 2023-11-24T06:27:24.847 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.841+0000 7f30757a9640 1 -- start start 2023-11-24T06:27:24.849 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f30757a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c003970 0x7f30701527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f30757a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30700c5d90 0x7f30700c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f30757a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30700c6730 0x7f30700b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f30757a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700c6bb0 con 0x7f305c003970 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f30757a9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700cb560 con 0x7f30700c6730 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f30757a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30700b4420 con 0x7f30700c5d90 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f306f7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30700c6730 0x7f30700b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f3067fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30700c5d90 0x7f30700c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f306f7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30700c6730 0x7f30700b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37322/0 (socket says 172.21.15.162:37322) 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.842+0000 7f306f7fe640 1 -- 172.21.15.162:0/2576831912 learned_addr learned my addr 172.21.15.162:0/2576831912 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306effd640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c003970 0x7f30701527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:24.850 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306f7fe640 1 -- 172.21.15.162:0/2576831912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30700c5d90 msgr2=0x7f30700c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306f7fe640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30700c5d90 0x7f30700c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306f7fe640 1 -- 172.21.15.162:0/2576831912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c003970 msgr2=0x7f30701527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306f7fe640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c003970 0x7f30701527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306f7fe640 1 -- 172.21.15.162:0/2576831912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700b4700 con 0x7f30700c6730 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306effd640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c003970 0x7f30701527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f306f7fe640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30700c6730 0x7f30700b51e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f30700709e0 tx=0x7f306800fc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f30677fe640 1 -- 172.21.15.162:0/2576831912 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800fec0 con 0x7f30700c6730 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f30677fe640 1 -- 172.21.15.162:0/2576831912 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3068006d60 con 0x7f30700c6730 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30700b49e0 con 0x7f30700c6730 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f30677fe640 1 -- 172.21.15.162:0/2576831912 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800d8a0 con 0x7f30700c6730 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.843+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30700c8510 con 0x7f30700c6730 2023-11-24T06:27:24.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.844+0000 7f30677fe640 1 -- 172.21.15.162:0/2576831912 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f306800dd00 con 0x7f30700c6730 2023-11-24T06:27:24.852 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.844+0000 7f30677fe640 1 -- 172.21.15.162:0/2576831912 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f3068007030 con 0x7f30700c6730 2023-11-24T06:27:24.852 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.845+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30700675d0 con 0x7f30700c6730 2023-11-24T06:27:24.852 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.847+0000 7f30677fe640 1 -- 172.21.15.162:0/2576831912 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f306801d460 con 0x7f30700c6730 2023-11-24T06:27:24.864 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:24.862+0000 7fd10de30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:27:24.888 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:24.885+0000 7f43f5246040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:27:24.927 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.925+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c", "id": 5} v 0) v1 -- 0x7f30700c7f80 con 0x7f30700c6730 2023-11-24T06:27:24.933 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.931+0000 7f30677fe640 1 -- 172.21.15.162:0/2576831912 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f306801d600 con 0x7f30700c6730 2023-11-24T06:27:24.933 INFO:teuthology.orchestra.run.smithi162.stdout:5 2023-11-24T06:27:24.934 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30700c6730 msgr2=0x7f30700b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:24.934 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30700c6730 0x7f30700b51e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f30700709e0 tx=0x7f306800fc70 comp rx=0 tx=0).stop 2023-11-24T06:27:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 shutdown_connections 2023-11-24T06:27:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30700c6730 0x7f30700b51e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30700c5d90 0x7f30700c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 --2- 172.21.15.162:0/2576831912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f305c003970 0x7f30701527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 >> 172.21.15.162:0/2576831912 conn(0x7f30700b2d00 msgr2=0x7f30700b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 shutdown_connections 2023-11-24T06:27:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:24.932+0000 7f30757a9640 1 -- 172.21.15.162:0/2576831912 wait complete. 2023-11-24T06:27:24.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new f592a84b-9a5e-4fc5-bc48-20ebf96843d4 6 2023-11-24T06:27:24.981 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:24.979+0000 7fd10de30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:27:25.004 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:25.002+0000 7f43f5246040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:27:25.101 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:25.098+0000 7fd10de30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:27:25.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.138+0000 7f56608d9640 1 Processor -- start 2023-11-24T06:27:25.141 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.138+0000 7f56608d9640 1 -- start start 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.140+0000 7f56608d9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c0cf9a0 0x7f565c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.140+0000 7f56608d9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f565c0b31d0 0x7f565c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.140+0000 7f56608d9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565c0d0320 con 0x7f565c15d4b0 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.140+0000 7f56608d9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c0d1c60 con 0x7f565c0b31d0 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.140+0000 7f56608d9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0b2730 con 0x7f565c0cf9a0 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.140+0000 7f565b59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f565c0b31d0 0x7f565c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565a59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c0cf9a0 0x7f565c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565b59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f565c0b31d0 0x7f565c0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37336/0 (socket says 172.21.15.162:37336) 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565a59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c0cf9a0 0x7f565c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:55088/0 (socket says 172.21.15.162:55088) 2023-11-24T06:27:25.143 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565b59e640 1 -- 172.21.15.162:0/1588676260 learned_addr learned my addr 172.21.15.162:0/1588676260 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:25.144 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565a59c640 1 -- 172.21.15.162:0/1588676260 >> v2:172.21.15.162:3300/0 conn(0x7f565c0b31d0 msgr2=0x7f565c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565a59c640 1 --2- 172.21.15.162:0/1588676260 >> v2:172.21.15.162:3300/0 conn(0x7f565c0b31d0 0x7f565c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565a59c640 1 -- 172.21.15.162:0/1588676260 >> v1:172.21.15.64:6789/0 conn(0x7f565c15d4b0 legacy=0x7f565c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565a59c640 1 -- 172.21.15.162:0/1588676260 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c0d1ec0 con 0x7f565c0cf9a0 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.141+0000 7f565b59e640 1 --2- 172.21.15.162:0/1588676260 >> v2:172.21.15.162:3300/0 conn(0x7f565c0b31d0 0x7f565c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.142+0000 7f565a59c640 1 --2- 172.21.15.162:0/1588676260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c0cf9a0 0x7f565c0cfde0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56500097d0 tx=0x7f565002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a24f1d196814d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.142+0000 7f5659d9b640 1 -- 172.21.15.162:0/1588676260 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650031780 con 0x7f565c0cf9a0 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.142+0000 7f5659d9b640 1 -- 172.21.15.162:0/1588676260 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56500318e0 con 0x7f565c0cf9a0 2023-11-24T06:27:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.142+0000 7f5659d9b640 1 -- 172.21.15.162:0/1588676260 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650031c70 con 0x7f565c0cf9a0 2023-11-24T06:27:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 -- 172.21.15.162:0/1588676260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c0cf9a0 msgr2=0x7f565c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 --2- 172.21.15.162:0/1588676260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c0cf9a0 0x7f565c0cfde0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56500097d0 tx=0x7f565002b820 comp rx=0 tx=0).stop 2023-11-24T06:27:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 -- 172.21.15.162:0/1588676260 shutdown_connections 2023-11-24T06:27:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 --2- 172.21.15.162:0/1588676260 >> v2:172.21.15.162:3300/0 conn(0x7f565c0b31d0 0x7f565c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 --2- 172.21.15.162:0/1588676260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c0cf9a0 0x7f565c0cfde0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 -- 172.21.15.162:0/1588676260 >> 172.21.15.162:0/1588676260 conn(0x7f565c0d3000 msgr2=0x7f565c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 -- 172.21.15.162:0/1588676260 shutdown_connections 2023-11-24T06:27:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.143+0000 7f56608d9640 1 -- 172.21.15.162:0/1588676260 wait complete. 2023-11-24T06:27:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.145+0000 7f56608d9640 1 Processor -- start 2023-11-24T06:27:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.145+0000 7f56608d9640 1 -- start start 2023-11-24T06:27:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.146+0000 7f56608d9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f565c0b31d0 0x7f565c1f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.146+0000 7f56608d9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f565c0cf9a0 0x7f565c152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.146+0000 7f56608d9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c15d4b0 0x7f565c15fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.146+0000 7f56608d9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0b6960 con 0x7f565c0b31d0 2023-11-24T06:27:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.146+0000 7f56608d9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0b5d50 con 0x7f565c0cf9a0 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.146+0000 7f56608d9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0b6030 con 0x7f565c15d4b0 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.146+0000 7f565a59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f565c0cf9a0 0x7f565c152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c15d4b0 0x7f565c15fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f565c0cf9a0 0x7f565c152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37340/0 (socket says 172.21.15.162:37340) 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 -- 172.21.15.162:0/3318495870 learned_addr learned my addr 172.21.15.162:0/3318495870 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565ad9d640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f565c0b31d0 0x7f565c1f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 -- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c15d4b0 msgr2=0x7f565c15fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c15d4b0 0x7f565c15fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 -- 172.21.15.162:0/3318495870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f565c0b31d0 msgr2=0x7f565c1f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f565c0b31d0 0x7f565c1f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 -- 172.21.15.162:0/3318495870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5654009b30 con 0x7f565c0cf9a0 2023-11-24T06:27:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565b59e640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c15d4b0 0x7f565c15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:25.151 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f565a59c640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f565c0cf9a0 0x7f565c152590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5650034040 tx=0x7f56500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.147+0000 7f56437fe640 1 -- 172.21.15.162:0/3318495870 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650032820 con 0x7f565c0cf9a0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.148+0000 7f56437fe640 1 -- 172.21.15.162:0/3318495870 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5650032980 con 0x7f565c0cf9a0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.148+0000 7f56437fe640 1 -- 172.21.15.162:0/3318495870 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650032d30 con 0x7f565c0cf9a0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.148+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5650009400 con 0x7f565c0cf9a0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.148+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c0b65f0 con 0x7f565c0cf9a0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.148+0000 7f56437fe640 1 -- 172.21.15.162:0/3318495870 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f565004e440 con 0x7f565c0cf9a0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.148+0000 7f56437fe640 1 -- 172.21.15.162:0/3318495870 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f565004ece0 con 0x7f565c0cf9a0 2023-11-24T06:27:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.149+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c08c830 con 0x7f565c0cf9a0 2023-11-24T06:27:25.153 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.151+0000 7f56437fe640 1 -- 172.21.15.162:0/3318495870 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f565004e6f0 con 0x7f565c0cf9a0 2023-11-24T06:27:25.172 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:25.169+0000 7f43f5246040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:27:25.224 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:25.221+0000 7f43f5246040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:27:25.233 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.230+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f592a84b-9a5e-4fc5-bc48-20ebf96843d4", "id": 6} v 0) v1 -- 0x7f565c08c830 con 0x7f565c0cf9a0 2023-11-24T06:27:25.243 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.241+0000 7f56437fe640 1 -- 172.21.15.162:0/3318495870 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f592a84b-9a5e-4fc5-bc48-20ebf96843d4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f565004e890 con 0x7f565c0cf9a0 2023-11-24T06:27:25.244 INFO:teuthology.orchestra.run.smithi162.stdout:6 2023-11-24T06:27:25.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.242+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f565c0cf9a0 msgr2=0x7f565c152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.242+0000 7f56608d9640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f565c0cf9a0 0x7f565c152590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5650034040 tx=0x7f56500046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.243+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 shutdown_connections 2023-11-24T06:27:25.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.243+0000 7f56608d9640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f565c15d4b0 0x7f565c15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.243+0000 7f56608d9640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f565c0cf9a0 0x7f565c152590 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.243+0000 7f56608d9640 1 --2- 172.21.15.162:0/3318495870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f565c0b31d0 0x7f565c1f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.243+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 >> 172.21.15.162:0/3318495870 conn(0x7f565c0d3000 msgr2=0x7f565c06ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:25.246 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.243+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 shutdown_connections 2023-11-24T06:27:25.246 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.243+0000 7f56608d9640 1 -- 172.21.15.162:0/3318495870 wait complete. 2023-11-24T06:27:25.261 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new fd4a964a-aa39-4a85-bb27-c3f06856b9af 7 2023-11-24T06:27:25.272 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:25.270+0000 7fd10de30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:27:25.324 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:25.322+0000 7fd10de30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:27:25.391 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:25.388+0000 7f43f5246040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:27:25.458 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.455+0000 7fb020be5640 1 Processor -- start 2023-11-24T06:27:25.458 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.455+0000 7fb020be5640 1 -- start start 2023-11-24T06:27:25.459 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb020be5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb01c0ce8f0 0x7fb01c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb020be5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb020be5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c0d0de0 con 0x7fb01c160f80 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb020be5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0d10c0 con 0x7fb01c0b2a00 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb020be5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0d13a0 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb013fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb01c0ce8f0 0x7fb01c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb013fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb01c0ce8f0 0x7fb01c0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37354/0 (socket says 172.21.15.162:37354) 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb013fff640 1 -- 172.21.15.162:0/1605922983 learned_addr learned my addr 172.21.15.162:0/1605922983 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb01ad9d640 1 --2- 172.21.15.162:0/1605922983 >> v2:172.21.15.64:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb013fff640 1 -- 172.21.15.162:0/1605922983 >> v2:172.21.15.64:3300/0 conn(0x7fb01c0b2a00 msgr2=0x7fb01c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:25.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb013fff640 1 --2- 172.21.15.162:0/1605922983 >> v2:172.21.15.64:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb013fff640 1 -- 172.21.15.162:0/1605922983 >> v1:172.21.15.64:6789/0 conn(0x7fb01c160f80 legacy=0x7fb01c161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:25.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.457+0000 7fb013fff640 1 -- 172.21.15.162:0/1605922983 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0d1680 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.458+0000 7fb013fff640 1 --2- 172.21.15.162:0/1605922983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c0ced30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0b5d10 tx=0x7fb00402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e8a27e3865994e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:25.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.459+0000 7fb019d9b640 1 -- 172.21.15.162:0/1605922983 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004031870 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.459+0000 7fb019d9b640 1 -- 172.21.15.162:0/1605922983 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0040319d0 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.462 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.459+0000 7fb019d9b640 1 -- 172.21.15.162:0/1605922983 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004031d60 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.462 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 -- 172.21.15.162:0/1605922983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 msgr2=0x7fb01c0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.462 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 --2- 172.21.15.162:0/1605922983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c0ced30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0b5d10 tx=0x7fb00402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 -- 172.21.15.162:0/1605922983 shutdown_connections 2023-11-24T06:27:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 --2- 172.21.15.162:0/1605922983 >> v2:172.21.15.64:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 --2- 172.21.15.162:0/1605922983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c0ced30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 -- 172.21.15.162:0/1605922983 >> 172.21.15.162:0/1605922983 conn(0x7fb01c09c550 msgr2=0x7fb01c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 -- 172.21.15.162:0/1605922983 shutdown_connections 2023-11-24T06:27:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.460+0000 7fb020be5640 1 -- 172.21.15.162:0/1605922983 wait complete. 2023-11-24T06:27:25.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.462+0000 7fb020be5640 1 Processor -- start 2023-11-24T06:27:25.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.463+0000 7fb020be5640 1 -- start start 2023-11-24T06:27:25.468 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.464+0000 7fb020be5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb01c0b2a00 0x7fb01c1f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.468 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.464+0000 7fb020be5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c1f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.464+0000 7fb020be5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c160f80 0x7fb01c1f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.464+0000 7fb020be5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0c6d40 con 0x7fb01c160f80 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.464+0000 7fb020be5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1f2b80 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.464+0000 7fb020be5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c15f740 con 0x7fb01c0b2a00 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c1f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c1f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37368/0 (socket says 172.21.15.162:37368) 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 -- 172.21.15.162:0/2506660339 learned_addr learned my addr 172.21.15.162:0/2506660339 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb01a59c640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb01c0b2a00 0x7fb01c1f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb01ad9d640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c160f80 0x7fb01c1f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 -- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb01c0b2a00 msgr2=0x7fb01c1f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb01c0b2a00 0x7fb01c1f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 -- 172.21.15.162:0/2506660339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c160f80 msgr2=0x7fb01c1f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c160f80 0x7fb01c1f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 -- 172.21.15.162:0/2506660339 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040094d0 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb01a59c640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb01c0b2a00 0x7fb01c1f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb01ad9d640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c160f80 0x7fb01c1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.465+0000 7fb013fff640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c1f6300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb00402b950 tx=0x7fb0040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.466+0000 7fb012ffd640 1 -- 172.21.15.162:0/2506660339 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004031860 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.466+0000 7fb012ffd640 1 -- 172.21.15.162:0/2506660339 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0040443f0 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.466+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c0c6f90 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.466+0000 7fb012ffd640 1 -- 172.21.15.162:0/2506660339 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0040447a0 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.466+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c0c7550 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.466+0000 7fb012ffd640 1 -- 172.21.15.162:0/2506660339 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb004044c00 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.466+0000 7fb012ffd640 1 -- 172.21.15.162:0/2506660339 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb00402d040 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.467+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0675d0 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.472 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.470+0000 7fb012ffd640 1 -- 172.21.15.162:0/2506660339 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb004035020 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.501 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:25.499+0000 7fd10de30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:27:25.558 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.555+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fd4a964a-aa39-4a85-bb27-c3f06856b9af", "id": 7} v 0) v1 -- 0x7fb01c0d3920 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.570 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.568+0000 7fb012ffd640 1 -- 172.21.15.162:0/2506660339 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fd4a964a-aa39-4a85-bb27-c3f06856b9af", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb00404db80 con 0x7fb01c0ce8f0 2023-11-24T06:27:25.570 INFO:teuthology.orchestra.run.smithi162.stdout:7 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 msgr2=0x7fb01c1f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c1f6300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb00402b950 tx=0x7fb0040046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 shutdown_connections 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c160f80 0x7fb01c1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c1f6300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 --2- 172.21.15.162:0/2506660339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb01c0b2a00 0x7fb01c1f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 >> 172.21.15.162:0/2506660339 conn(0x7fb01c09c550 msgr2=0x7fb01c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 shutdown_connections 2023-11-24T06:27:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:27:25.569+0000 7fb020be5640 1 -- 172.21.15.162:0/2506660339 wait complete. 2023-11-24T06:27:25.577 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:25.585 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:25.591 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T06:27:25.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T06:27:25.632 INFO:tasks.ceph.osd.0:Started 2023-11-24T06:27:25.632 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T06:27:25.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T06:27:25.637 INFO:tasks.ceph.osd.1:Started 2023-11-24T06:27:25.638 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T06:27:25.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T06:27:25.645 INFO:tasks.ceph.osd.2:Started 2023-11-24T06:27:25.645 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T06:27:25.645 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T06:27:25.651 INFO:tasks.ceph.osd.3:Started 2023-11-24T06:27:25.651 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T06:27:25.651 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T06:27:25.655 INFO:tasks.ceph.osd.4:Started 2023-11-24T06:27:25.656 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T06:27:25.656 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T06:27:25.661 INFO:tasks.ceph.osd.5:Started 2023-11-24T06:27:25.661 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T06:27:25.661 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T06:27:25.667 INFO:tasks.ceph.osd.6:Started 2023-11-24T06:27:25.667 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T06:27:25.668 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T06:27:25.672 INFO:tasks.ceph.osd.7:Started 2023-11-24T06:27:25.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:25.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.894+0000 7f224301e640 1 Processor -- start 2023-11-24T06:27:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.894+0000 7f224301e640 1 -- start start 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.895+0000 7f224301e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.895+0000 7f224301e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223c15c5f0 con 0x7f223c13e360 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f224301e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223c06c470 con 0x7f223c13df80 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f224301e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c06c750 con 0x7f223c0cf9a0 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f223bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f223c13e360 0x7f223c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57882/0 (socket says 172.21.15.64:57882) 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f223bfff640 1 -- 172.21.15.64:0/3772658248 learned_addr learned my addr 172.21.15.64:0/3772658248 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f22415bb640 1 --2- 172.21.15.64:0/3772658248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f223b7fe640 1 -- 172.21.15.64:0/3772658248 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279204981 0 0) 0x7f223c06c470 con 0x7f223c13df80 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f223b7fe640 1 -- 172.21.15.64:0/3772658248 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f222c003520 con 0x7f223c13df80 2023-11-24T06:27:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f223b7fe640 1 -- 172.21.15.64:0/3772658248 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593579213 0 0) 0x7f223c15c5f0 con 0x7f223c13e360 2023-11-24T06:27:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f223b7fe640 1 -- 172.21.15.64:0/3772658248 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f223c06c470 con 0x7f223c13e360 2023-11-24T06:27:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f22415bb640 1 -- 172.21.15.64:0/3772658248 >> v1:172.21.15.162:6789/0 conn(0x7f223c13df80 legacy=0x7f223c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f22415bb640 1 -- 172.21.15.64:0/3772658248 >> v1:172.21.15.64:6789/0 conn(0x7f223c13e360 legacy=0x7f223c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.896+0000 7f22415bb640 1 -- 172.21.15.64:0/3772658248 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c06ca30 con 0x7f223c0cf9a0 2023-11-24T06:27:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.897+0000 7f22415bb640 1 --2- 172.21.15.64:0/3772658248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c166030 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2234009dc0 tx=0x7f223402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535b8d8f71e8b615 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f223b7fe640 1 -- 172.21.15.64:0/3772658248 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223402ccd0 con 0x7f223c0cf9a0 2023-11-24T06:27:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f223b7fe640 1 -- 172.21.15.64:0/3772658248 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223402ce30 con 0x7f223c0cf9a0 2023-11-24T06:27:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f223b7fe640 1 -- 172.21.15.64:0/3772658248 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234034660 con 0x7f223c0cf9a0 2023-11-24T06:27:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f224301e640 1 -- 172.21.15.64:0/3772658248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 msgr2=0x7f223c166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f224301e640 1 --2- 172.21.15.64:0/3772658248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c166030 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2234009dc0 tx=0x7f223402bf20 comp rx=0 tx=0).stop 2023-11-24T06:27:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f224301e640 1 -- 172.21.15.64:0/3772658248 shutdown_connections 2023-11-24T06:27:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f224301e640 1 --2- 172.21.15.64:0/3772658248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c166030 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.898+0000 7f224301e640 1 -- 172.21.15.64:0/3772658248 >> 172.21.15.64:0/3772658248 conn(0x7f223c06a800 msgr2=0x7f223c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.899+0000 7f224301e640 1 -- 172.21.15.64:0/3772658248 shutdown_connections 2023-11-24T06:27:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.899+0000 7f224301e640 1 -- 172.21.15.64:0/3772658248 wait complete. 2023-11-24T06:27:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.900+0000 7f224301e640 1 Processor -- start 2023-11-24T06:27:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.900+0000 7f224301e640 1 -- start start 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f224301e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f224301e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223c13df80 0x7f223c0add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f224301e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f223c13e360 0x7f223c0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f224301e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0ae2a0 con 0x7f223c13e360 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f224301e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c1f0f90 con 0x7f223c13df80 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f224301e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0a9bf0 con 0x7f223c0cf9a0 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f22415bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f223c13e360 0x7f223c0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f223bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223c13df80 0x7f223c0add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.901+0000 7f223bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223c13df80 0x7f223c0add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:52392/0 (socket says 172.21.15.64:52392) 2023-11-24T06:27:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f223bfff640 1 -- 172.21.15.64:0/3747302291 learned_addr learned my addr 172.21.15.64:0/3747302291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f2240dba640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f223bfff640 1 -- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 msgr2=0x7f223c158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f223bfff640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f223bfff640 1 -- 172.21.15.64:0/3747302291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f223c13e360 msgr2=0x7f223c0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f223bfff640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f223c13e360 0x7f223c0b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f223bfff640 1 -- 172.21.15.64:0/3747302291 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234009a70 con 0x7f223c13df80 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f223bfff640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223c13df80 0x7f223c0add60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f222800c500 tx=0x7f222800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.902+0000 7f22397fa640 1 -- 172.21.15.64:0/3747302291 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228005ce0 con 0x7f223c13df80 2023-11-24T06:27:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.903+0000 7f22397fa640 1 -- 172.21.15.64:0/3747302291 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2228005e40 con 0x7f223c13df80 2023-11-24T06:27:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.903+0000 7f22397fa640 1 -- 172.21.15.64:0/3747302291 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228007310 con 0x7f223c13df80 2023-11-24T06:27:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.903+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c157cb0 con 0x7f223c13df80 2023-11-24T06:27:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.903+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c158270 con 0x7f223c13df80 2023-11-24T06:27:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.903+0000 7f22397fa640 1 -- 172.21.15.64:0/3747302291 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2228012400 con 0x7f223c13df80 2023-11-24T06:27:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.903+0000 7f22397fa640 1 -- 172.21.15.64:0/3747302291 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f222801b360 con 0x7f223c13df80 2023-11-24T06:27:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.904+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c158490 con 0x7f223c13df80 2023-11-24T06:27:25.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.906+0000 7f22397fa640 1 -- 172.21.15.64:0/3747302291 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2228018070 con 0x7f223c13df80 2023-11-24T06:27:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.972+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f223c0675d0 con 0x7f223c13df80 2023-11-24T06:27:25.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.974+0000 7f22397fa640 1 -- 172.21.15.64:0/3747302291 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2228012710 con 0x7f223c13df80 2023-11-24T06:27:25.977 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:25.978 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:25.558077+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223c13df80 msgr2=0x7f223c0add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223c13df80 0x7f223c0add60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f222800c500 tx=0x7f222800c9c0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 shutdown_connections 2023-11-24T06:27:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f223c13e360 0x7f223c0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223c13df80 0x7f223c0add60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 --2- 172.21.15.64:0/3747302291 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f223c0cf9a0 0x7f223c158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 >> 172.21.15.64:0/3747302291 conn(0x7f223c06a800 msgr2=0x7f223c15ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 shutdown_connections 2023-11-24T06:27:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:25.975+0000 7f224301e640 1 -- 172.21.15.64:0/3747302291 wait complete. 2023-11-24T06:27:25.994 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T06:27:25.994 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T06:27:26.352 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-11-24T06:27:26.349+0000 7f647bc14500 -1 Falling back to public interface 2023-11-24T06:27:26.352 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-11-24T06:27:26.349+0000 7fa5947be500 -1 Falling back to public interface 2023-11-24T06:27:26.352 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-11-24T06:27:26.349+0000 7f647a5d3640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-11-24T06:27:26.353 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-11-24T06:27:26.350+0000 7f922b013500 -1 Falling back to public interface 2023-11-24T06:27:26.353 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-11-24T06:27:26.351+0000 7f922996c640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6806/0: (98) Address already in use 2023-11-24T06:27:26.355 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-11-24T06:27:26.353+0000 7fa593767640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6811/0: (98) Address already in use 2023-11-24T06:27:26.356 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T06:27:26.353+0000 7f8359e12500 -1 Falling back to public interface 2023-11-24T06:27:26.359 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-11-24T06:27:26.356+0000 7f922996c640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6818/0: (98) Address already in use 2023-11-24T06:27:26.376 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T06:27:26.373+0000 7f807ac11500 -1 Falling back to public interface 2023-11-24T06:27:26.383 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T06:27:26.381+0000 7f109d438500 -1 Falling back to public interface 2023-11-24T06:27:26.393 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T06:27:26.391+0000 7f2877ada500 -1 Falling back to public interface 2023-11-24T06:27:26.400 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-11-24T06:27:26.398+0000 7fed1b80c500 -1 Falling back to public interface 2023-11-24T06:27:26.564 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:26.561+0000 7f43f5246040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:27:26.657 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:26.655+0000 7fd10de30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:27:26.800 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:26.886 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:26.883+0000 7f43f5246040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:27:26.950 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:26.948+0000 7f43f5246040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:27:26.967 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:26.965+0000 7fd10de30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:27:26.968 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.965+0000 7ffb18f90640 1 Processor -- start 2023-11-24T06:27:26.968 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.966+0000 7ffb18f90640 1 -- start start 2023-11-24T06:27:26.970 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb18f90640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb14160c80 0x7ffb140cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:26.970 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb18f90640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb140ce3a0 0x7ffb140d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:26.970 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb18f90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140ce860 con 0x7ffb140d2030 2023-11-24T06:27:26.970 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb18f90640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140b3230 con 0x7ffb14160c80 2023-11-24T06:27:26.970 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb18f90640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140ceb40 con 0x7ffb140ce3a0 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb1259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb14160c80 0x7ffb140cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb1259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb14160c80 0x7ffb140cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55634/0 (socket says 172.21.15.64:55634) 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb1259c640 1 -- 172.21.15.64:0/2282477211 learned_addr learned my addr 172.21.15.64:0/2282477211 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.967+0000 7ffb0bfff640 1 --2- 172.21.15.64:0/2282477211 >> v2:172.21.15.162:3300/0 conn(0x7ffb140ce3a0 0x7ffb140d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.968+0000 7ffb1259c640 1 -- 172.21.15.64:0/2282477211 >> v2:172.21.15.162:3300/0 conn(0x7ffb140ce3a0 msgr2=0x7ffb140d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.968+0000 7ffb1259c640 1 --2- 172.21.15.64:0/2282477211 >> v2:172.21.15.162:3300/0 conn(0x7ffb140ce3a0 0x7ffb140d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.968+0000 7ffb1259c640 1 -- 172.21.15.64:0/2282477211 >> v1:172.21.15.162:6789/0 conn(0x7ffb140d2030 legacy=0x7ffb140cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:26.971 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.968+0000 7ffb1259c640 1 -- 172.21.15.64:0/2282477211 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140cee20 con 0x7ffb14160c80 2023-11-24T06:27:26.973 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.969+0000 7ffb1259c640 1 --2- 172.21.15.64:0/2282477211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 0x7ffb140cde60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb140b5de0 tx=0x7ffb0402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1a07760557eaa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:26.973 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.969+0000 7ffb11d9b640 1 -- 172.21.15.64:0/2282477211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04031870 con 0x7ffb14160c80 2023-11-24T06:27:26.973 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.969+0000 7ffb11d9b640 1 -- 172.21.15.64:0/2282477211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb040319d0 con 0x7ffb14160c80 2023-11-24T06:27:26.973 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.969+0000 7ffb11d9b640 1 -- 172.21.15.64:0/2282477211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04031d60 con 0x7ffb14160c80 2023-11-24T06:27:26.973 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.969+0000 7ffb18f90640 1 -- 172.21.15.64:0/2282477211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 msgr2=0x7ffb140cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:26.973 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.969+0000 7ffb18f90640 1 --2- 172.21.15.64:0/2282477211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 0x7ffb140cde60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb140b5de0 tx=0x7ffb0402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:26.973 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.970+0000 7ffb18f90640 1 -- 172.21.15.64:0/2282477211 shutdown_connections 2023-11-24T06:27:26.974 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.970+0000 7ffb18f90640 1 --2- 172.21.15.64:0/2282477211 >> v2:172.21.15.162:3300/0 conn(0x7ffb140ce3a0 0x7ffb140d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:26.974 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.970+0000 7ffb18f90640 1 --2- 172.21.15.64:0/2282477211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 0x7ffb140cde60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:26.974 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.970+0000 7ffb18f90640 1 -- 172.21.15.64:0/2282477211 >> 172.21.15.64:0/2282477211 conn(0x7ffb1408aff0 msgr2=0x7ffb1409c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:26.974 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.970+0000 7ffb18f90640 1 -- 172.21.15.64:0/2282477211 shutdown_connections 2023-11-24T06:27:26.974 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.970+0000 7ffb18f90640 1 -- 172.21.15.64:0/2282477211 wait complete. 2023-11-24T06:27:26.976 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.971+0000 7ffb18f90640 1 Processor -- start 2023-11-24T06:27:26.977 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.972+0000 7ffb18f90640 1 -- start start 2023-11-24T06:27:26.977 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb18f90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb140ce3a0 0x7ffb141f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:26.977 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb18f90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb140d2030 0x7ffb140c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:26.977 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb18f90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 0x7ffb140cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:26.977 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb18f90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c97b0 con 0x7ffb14160c80 2023-11-24T06:27:26.977 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb18f90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c9c40 con 0x7ffb140d2030 2023-11-24T06:27:26.977 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb18f90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c8ba0 con 0x7ffb140ce3a0 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb12d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 0x7ffb140cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb140d2030 0x7ffb140c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb140d2030 0x7ffb140c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:52396/0 (socket says 172.21.15.64:52396) 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 -- 172.21.15.64:0/926278847 learned_addr learned my addr 172.21.15.64:0/926278847 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb1259c640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb140ce3a0 0x7ffb141f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 -- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb140ce3a0 msgr2=0x7ffb141f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb140ce3a0 0x7ffb141f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:26.978 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 -- 172.21.15.64:0/926278847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 msgr2=0x7ffb140cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:26.979 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 0x7ffb140cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:26.979 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.973+0000 7ffb0bfff640 1 -- 172.21.15.64:0/926278847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040094d0 con 0x7ffb140d2030 2023-11-24T06:27:26.979 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb0bfff640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb140d2030 0x7ffb140c82a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00d5f0 tx=0x7ffafc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:26.979 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb0affd640 1 -- 172.21.15.64:0/926278847 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0125f0 con 0x7ffb140d2030 2023-11-24T06:27:26.979 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb0affd640 1 -- 172.21.15.64:0/926278847 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc012750 con 0x7ffb140d2030 2023-11-24T06:27:26.979 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb0affd640 1 -- 172.21.15.64:0/926278847 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc012b00 con 0x7ffb140d2030 2023-11-24T06:27:26.979 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140c8e80 con 0x7ffb140d2030 2023-11-24T06:27:26.980 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140c9440 con 0x7ffb140d2030 2023-11-24T06:27:26.981 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb0affd640 1 -- 172.21.15.64:0/926278847 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffafc01e460 con 0x7ffb140d2030 2023-11-24T06:27:26.981 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.974+0000 7ffb0affd640 1 -- 172.21.15.64:0/926278847 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ffafc014c30 con 0x7ffb140d2030 2023-11-24T06:27:26.981 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.975+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1406cf20 con 0x7ffb140d2030 2023-11-24T06:27:26.981 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:26.978+0000 7ffb0affd640 1 -- 172.21.15.64:0/926278847 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ffafc014e40 con 0x7ffb140d2030 2023-11-24T06:27:27.029 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:27.026+0000 7fd10de30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:27:27.045 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.042+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb140c9920 con 0x7ffb140d2030 2023-11-24T06:27:27.046 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.044+0000 7ffb0affd640 1 -- 172.21.15.64:0/926278847 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ffafc01e8a0 con 0x7ffb140d2030 2023-11-24T06:27:27.046 INFO:teuthology.misc.health.smithi064.stdout: 2023-11-24T06:27:27.047 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":10,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:25.558077+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:27.048 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb140d2030 msgr2=0x7ffb140c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.048 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb140d2030 0x7ffb140c82a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00d5f0 tx=0x7ffafc00dab0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.048 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 shutdown_connections 2023-11-24T06:27:27.048 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb14160c80 0x7ffb140cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.048 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb140d2030 0x7ffb140c82a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.048 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 --2- 172.21.15.64:0/926278847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb140ce3a0 0x7ffb141f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.049 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 >> 172.21.15.64:0/926278847 conn(0x7ffb1408aff0 msgr2=0x7ffb1406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:27.049 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 shutdown_connections 2023-11-24T06:27:27.049 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:27.045+0000 7ffb18f90640 1 -- 172.21.15.64:0/926278847 wait complete. 2023-11-24T06:27:27.060 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:27:27.072 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:27.070+0000 7f43f5246040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:27:27.149 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-11-24T06:27:27.146+0000 7fd10de30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:27:28.108 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:28.105+0000 7f4396582640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T06:27:28.111 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T06:27:28.108+0000 7f4396582640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T06:27:29.941 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-11-24T06:27:29.939+0000 7f647bc14500 -1 osd.4 0 log_to_monitors true 2023-11-24T06:27:29.969 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T06:27:29.966+0000 7f109d438500 -1 osd.2 0 log_to_monitors true 2023-11-24T06:27:29.973 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T06:27:29.971+0000 7f2877ada500 -1 osd.0 0 log_to_monitors true 2023-11-24T06:27:29.984 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-11-24T06:27:29.981+0000 7fa5947be500 -1 osd.6 0 log_to_monitors true 2023-11-24T06:27:29.987 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-11-24T06:27:29.985+0000 7f922b013500 -1 osd.5 0 log_to_monitors true 2023-11-24T06:27:29.999 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-11-24T06:27:29.997+0000 7fed1b80c500 -1 osd.7 0 log_to_monitors true 2023-11-24T06:27:30.151 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T06:27:30.149+0000 7f8359e12500 -1 osd.3 0 log_to_monitors true 2023-11-24T06:27:30.190 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T06:27:30.187+0000 7f807ac11500 -1 osd.1 0 log_to_monitors true 2023-11-24T06:27:32.837 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T06:27:32.834+0000 7f109a3dd640 -1 osd.2 0 waiting for initial osdmap 2023-11-24T06:27:32.851 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T06:27:32.849+0000 7f2874a7d640 -1 osd.0 0 waiting for initial osdmap 2023-11-24T06:27:32.857 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T06:27:32.855+0000 7f10911de640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:32.878 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T06:27:32.876+0000 7f286b87e640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:33.405 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-11-24T06:27:33.402+0000 7f6476d97640 -1 osd.4 0 waiting for initial osdmap 2023-11-24T06:27:33.415 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-11-24T06:27:33.413+0000 7f646f3d0640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:33.460 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-11-24T06:27:33.458+0000 7fa58ff2b640 -1 osd.6 0 waiting for initial osdmap 2023-11-24T06:27:33.474 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-11-24T06:27:33.472+0000 7fa588564640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:33.486 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-11-24T06:27:33.483+0000 7fed1692e640 -1 osd.7 0 waiting for initial osdmap 2023-11-24T06:27:33.500 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-11-24T06:27:33.498+0000 7fed0ef67640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:33.522 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-11-24T06:27:33.519+0000 7f9226130640 -1 osd.5 0 waiting for initial osdmap 2023-11-24T06:27:33.532 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-11-24T06:27:33.530+0000 7f921e769640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:33.744 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T06:27:33.741+0000 7f8075e4c640 -1 osd.1 0 waiting for initial osdmap 2023-11-24T06:27:33.752 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T06:27:33.749+0000 7f806e485640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:33.787 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T06:27:33.784+0000 7f8354ee6640 -1 osd.3 0 waiting for initial osdmap 2023-11-24T06:27:33.805 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T06:27:33.802+0000 7f834d51f640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:27:33.867 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:34.030 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.028+0000 7fefd5c51640 1 Processor -- start 2023-11-24T06:27:34.031 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.028+0000 7fefd5c51640 1 -- start start 2023-11-24T06:27:34.033 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.029+0000 7fefd5c51640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd013cd20 0x7fefd015e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.033 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.029+0000 7fefd5c51640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefd013d100 0x7fefd006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.033 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.029+0000 7fefd5c51640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd015b5b0 con 0x7fefd006c720 2023-11-24T06:27:34.033 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefd5c51640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd006cb60 con 0x7fefd013d100 2023-11-24T06:27:34.033 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefd5c51640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd015b2d0 con 0x7fefd013cd20 2023-11-24T06:27:34.033 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefc7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefd013d100 0x7fefd006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.033 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefc7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefd013d100 0x7fefd006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52846/0 (socket says 172.21.15.64:52846) 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefc7fff640 1 -- 172.21.15.64:0/3490520918 learned_addr learned my addr 172.21.15.64:0/3490520918 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefcf7fe640 1 --2- 172.21.15.64:0/3490520918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd013cd20 0x7fefd015e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefc7fff640 1 -- 172.21.15.64:0/3490520918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd013cd20 msgr2=0x7fefd015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefc7fff640 1 --2- 172.21.15.64:0/3490520918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd013cd20 0x7fefd015e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefc7fff640 1 -- 172.21.15.64:0/3490520918 >> v1:172.21.15.64:6789/0 conn(0x7fefd006c720 legacy=0x7fefd015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefceffd640 1 -- 172.21.15.64:0/3490520918 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299757316 0 0) 0x7fefd015b5b0 con 0x7fefd006c720 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefc7fff640 1 -- 172.21.15.64:0/3490520918 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd015dd30 con 0x7fefd013d100 2023-11-24T06:27:34.034 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.030+0000 7fefcf7fe640 1 --2- 172.21.15.64:0/3490520918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd013cd20 0x7fefd015e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:34.036 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.031+0000 7fefc7fff640 1 --2- 172.21.15.64:0/3490520918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd006c1e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009870 tx=0x7fefb802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39de80c996734ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.036 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefceffd640 1 -- 172.21.15.64:0/3490520918 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8031640 con 0x7fefd013d100 2023-11-24T06:27:34.036 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefceffd640 1 -- 172.21.15.64:0/3490520918 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb80317a0 con 0x7fefd013d100 2023-11-24T06:27:34.036 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefceffd640 1 -- 172.21.15.64:0/3490520918 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8031b30 con 0x7fefd013d100 2023-11-24T06:27:34.036 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefd5c51640 1 -- 172.21.15.64:0/3490520918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 msgr2=0x7fefd006c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.036 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3490520918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd006c1e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009870 tx=0x7fefb802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.036 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefd5c51640 1 -- 172.21.15.64:0/3490520918 shutdown_connections 2023-11-24T06:27:34.037 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3490520918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd006c1e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.037 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3490520918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd013cd20 0x7fefd015e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.037 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefd5c51640 1 -- 172.21.15.64:0/3490520918 >> 172.21.15.64:0/3490520918 conn(0x7fefd006dd50 msgr2=0x7fefd006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.037 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.032+0000 7fefd5c51640 1 -- 172.21.15.64:0/3490520918 shutdown_connections 2023-11-24T06:27:34.037 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.033+0000 7fefd5c51640 1 -- 172.21.15.64:0/3490520918 wait complete. 2023-11-24T06:27:34.039 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.034+0000 7fefd5c51640 1 Processor -- start 2023-11-24T06:27:34.040 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.034+0000 7fefd5c51640 1 -- start start 2023-11-24T06:27:34.040 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.035+0000 7fefd5c51640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd006c720 0x7fefd01388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.040 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.035+0000 7fefd5c51640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefd013cd20 0x7fefd00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.040 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.035+0000 7fefd5c51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd00b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.040 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.035+0000 7fefd5c51640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd013d520 con 0x7fefd013d100 2023-11-24T06:27:34.041 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.035+0000 7fefd5c51640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd01f64e0 con 0x7fefd013cd20 2023-11-24T06:27:34.041 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.035+0000 7fefd5c51640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd00b1090 con 0x7fefd006c720 2023-11-24T06:27:34.041 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd00b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd00b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52852/0 (socket says 172.21.15.64:52852) 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 -- 172.21.15.64:0/3306723931 learned_addr learned my addr 172.21.15.64:0/3306723931 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcf7fe640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd006c720 0x7fefd01388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefc7fff640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefd013cd20 0x7fefd00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 -- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd006c720 msgr2=0x7fefd01388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd006c720 0x7fefd01388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 -- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefd013cd20 msgr2=0x7fefd00b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefd013cd20 0x7fefd00b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 -- 172.21.15.64:0/3306723931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80094d0 con 0x7fefd013d100 2023-11-24T06:27:34.042 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.036+0000 7fefcffff640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd00b0b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefc8001560 tx=0x7fefc800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.043 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.037+0000 7fefccff9640 1 -- 172.21.15.64:0/3306723931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8003da0 con 0x7fefd013d100 2023-11-24T06:27:34.043 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.037+0000 7fefccff9640 1 -- 172.21.15.64:0/3306723931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8005070 con 0x7fefd013d100 2023-11-24T06:27:34.043 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.037+0000 7fefccff9640 1 -- 172.21.15.64:0/3306723931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8005420 con 0x7fefd013d100 2023-11-24T06:27:34.043 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.037+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd01f6730 con 0x7fefd013d100 2023-11-24T06:27:34.043 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.037+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd01f6cf0 con 0x7fefd013d100 2023-11-24T06:27:34.044 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.038+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd00675d0 con 0x7fefd013d100 2023-11-24T06:27:34.044 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.039+0000 7fefccff9640 1 -- 172.21.15.64:0/3306723931 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fefc8005580 con 0x7fefd013d100 2023-11-24T06:27:34.044 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.040+0000 7fefccff9640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fefb006ae00 0x7fefb006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.044 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.040+0000 7fefccff9640 1 -- 172.21.15.64:0/3306723931 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fefc80888c0 con 0x7fefd013d100 2023-11-24T06:27:34.044 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.040+0000 7fefcf7fe640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fefb006ae00 0x7fefb006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.044 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.041+0000 7fefcf7fe640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fefb006ae00 0x7fefb006d2c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fefc000a990 tx=0x7fefc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.047 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.045+0000 7fefccff9640 1 -- 172.21.15.64:0/3306723931 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fefc8053f90 con 0x7fefd013d100 2023-11-24T06:27:34.241 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.238+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefd006ad40 con 0x7fefd013d100 2023-11-24T06:27:34.242 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.240+0000 7fefccff9640 1 -- 172.21.15.64:0/3306723931 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10884 (secure 0 0 0) 0x7fefd006ad40 con 0x7fefd013d100 2023-11-24T06:27:34.242 INFO:teuthology.misc.health.smithi064.stdout: 2023-11-24T06:27:34.243 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":14,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:33.849089+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:34.245 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fefb006ae00 msgr2=0x7fefb006d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.245 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fefb006ae00 0x7fefb006d2c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fefc000a990 tx=0x7fefc0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:34.245 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 msgr2=0x7fefd00b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.245 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd00b0b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefc8001560 tx=0x7fefc800cc70 comp rx=0 tx=0).stop 2023-11-24T06:27:34.245 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 shutdown_connections 2023-11-24T06:27:34.245 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fefb006ae00 0x7fefb006d2c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.245 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd013d100 0x7fefd00b0b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.246 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefd013cd20 0x7fefd00b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.246 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 --2- 172.21.15.64:0/3306723931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefd006c720 0x7fefd01388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.246 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 >> 172.21.15.64:0/3306723931 conn(0x7fefd006dd50 msgr2=0x7fefd015a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.246 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.242+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 shutdown_connections 2023-11-24T06:27:34.246 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T06:27:34.243+0000 7fefd5c51640 1 -- 172.21.15.64:0/3306723931 wait complete. 2023-11-24T06:27:34.259 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T06:27:34.259 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T06:27:34.260 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:27:34.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:27:34.268 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:27:34.268 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:27:34.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:27:34.278 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:27:34.278 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:27:34.278 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:27:34.288 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:27:34.288 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:27:34.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:27:34.296 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:27:34.297 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T06:27:34.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:34.421 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T06:27:34.422 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T06:27:34.450 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2023-11-24T06:27:34.457 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2023-11-24T06:27:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.477+0000 7f28a0d67640 1 Processor -- start 2023-11-24T06:27:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.477+0000 7f28a0d67640 1 -- start start 2023-11-24T06:27:34.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f28a0d67640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f289c160920 0x7f289c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f28a0d67640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cfea0 0x7f289c161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f28a0d67640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289c0d0a10 con 0x7f289c0cf9f0 2023-11-24T06:27:34.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f28a0d67640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289c0d0330 con 0x7f289c160920 2023-11-24T06:27:34.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f28a0d67640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0d1f50 con 0x7f289c0cfea0 2023-11-24T06:27:34.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f289c160920 0x7f289c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f289c160920 0x7f289c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52908/0 (socket says 172.21.15.64:52908) 2023-11-24T06:27:34.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289b7fe640 1 -- 172.21.15.64:0/4116305689 learned_addr learned my addr 172.21.15.64:0/4116305689 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:34.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cfea0 0x7f289c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289b7fe640 1 -- 172.21.15.64:0/4116305689 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cfea0 msgr2=0x7f289c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289b7fe640 1 --2- 172.21.15.64:0/4116305689 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cfea0 0x7f289c161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289b7fe640 1 -- 172.21.15.64:0/4116305689 >> v1:172.21.15.162:6789/0 conn(0x7f289c0cf9f0 legacy=0x7f289c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.479+0000 7f289b7fe640 1 -- 172.21.15.64:0/4116305689 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c0cf080 con 0x7f289c160920 2023-11-24T06:27:34.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.480+0000 7f289b7fe640 1 --2- 172.21.15.64:0/4116305689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c160920 0x7f289c0b3f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f288c009870 tx=0x7f288c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa44b61d01114e3b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f289affd640 1 -- 172.21.15.64:0/4116305689 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c030840 con 0x7f289c160920 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f289affd640 1 -- 172.21.15.64:0/4116305689 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c0309a0 con 0x7f289c160920 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f289affd640 1 -- 172.21.15.64:0/4116305689 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c030d30 con 0x7f289c160920 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 -- 172.21.15.64:0/4116305689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c160920 msgr2=0x7f289c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 --2- 172.21.15.64:0/4116305689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c160920 0x7f289c0b3f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f288c009870 tx=0x7f288c02b980 comp rx=0 tx=0).stop 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 -- 172.21.15.64:0/4116305689 shutdown_connections 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 --2- 172.21.15.64:0/4116305689 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cfea0 0x7f289c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 --2- 172.21.15.64:0/4116305689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c160920 0x7f289c0b3f30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 -- 172.21.15.64:0/4116305689 >> 172.21.15.64:0/4116305689 conn(0x7f289c0d3090 msgr2=0x7f289c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 -- 172.21.15.64:0/4116305689 shutdown_connections 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.481+0000 7f28a0d67640 1 -- 172.21.15.64:0/4116305689 wait complete. 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.483+0000 7f28a0d67640 1 Processor -- start 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.483+0000 7f28a0d67640 1 -- start start 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.484+0000 7f28a0d67640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cf9f0 0x7f289c0b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.484+0000 7f28a0d67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cfea0 0x7f289c0b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.484+0000 7f28a0d67640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f289c160920 0x7f289c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.484+0000 7f28a0d67640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0b6b80 con 0x7f289c0cfea0 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.484+0000 7f28a0d67640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0b73e0 con 0x7f289c160920 2023-11-24T06:27:34.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.484+0000 7f28a0d67640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0b76c0 con 0x7f289c0cf9f0 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.484+0000 7f2893fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cfea0 0x7f289c0b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f289b7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cf9f0 0x7f289c0b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cfea0 0x7f289c0b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52924/0 (socket says 172.21.15.64:52924) 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 -- 172.21.15.64:0/2792605102 learned_addr learned my addr 172.21.15.64:0/2792605102 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f289bfff640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f289c160920 0x7f289c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 -- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cf9f0 msgr2=0x7f289c0b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cf9f0 0x7f289c0b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 -- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f289c160920 msgr2=0x7f289c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f289c160920 0x7f289c0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 -- 172.21.15.64:0/2792605102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0094d0 con 0x7f289c0cfea0 2023-11-24T06:27:34.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f289b7fe640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cf9f0 0x7f289c0b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:34.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f289bfff640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f289c160920 0x7f289c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:34.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2893fff640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cfea0 0x7f289c0b8480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f288400c400 tx=0x7f288400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.485+0000 7f2898ff9640 1 -- 172.21.15.64:0/2792605102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28840052d0 con 0x7f289c0cfea0 2023-11-24T06:27:34.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.486+0000 7f2898ff9640 1 -- 172.21.15.64:0/2792605102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2884005430 con 0x7f289c0cfea0 2023-11-24T06:27:34.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.486+0000 7f2898ff9640 1 -- 172.21.15.64:0/2792605102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884003830 con 0x7f289c0cfea0 2023-11-24T06:27:34.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.486+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c0b79a0 con 0x7f289c0cfea0 2023-11-24T06:27:34.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.486+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c0b7f60 con 0x7f289c0cfea0 2023-11-24T06:27:34.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.487+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c0d3fa0 con 0x7f289c0cfea0 2023-11-24T06:27:34.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.488+0000 7f2898ff9640 1 -- 172.21.15.64:0/2792605102 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f2884003990 con 0x7f289c0cfea0 2023-11-24T06:27:34.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.488+0000 7f2898ff9640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f287c06adb0 0x7f287c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.489+0000 7f2898ff9640 1 -- 172.21.15.64:0/2792605102 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f2884088580 con 0x7f289c0cfea0 2023-11-24T06:27:34.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.489+0000 7f289b7fe640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f287c06adb0 0x7f287c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.489+0000 7f289b7fe640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f287c06adb0 0x7f287c06d270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f288c036000 tx=0x7f288c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.493+0000 7f2898ff9640 1 -- 172.21.15.64:0/2792605102 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2884053bf0 con 0x7f289c0cfea0 2023-11-24T06:27:34.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.689+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f289c067450 con 0x7f289c0cfea0 2023-11-24T06:27:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.691+0000 7f2898ff9640 1 -- 172.21.15.64:0/2792605102 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10884 (secure 0 0 0) 0x7f2884017090 con 0x7f289c0cfea0 2023-11-24T06:27:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:34.694 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:33.849089+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f287c06adb0 msgr2=0x7f287c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f287c06adb0 0x7f287c06d270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f288c036000 tx=0x7f288c008030 comp rx=0 tx=0).stop 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cfea0 msgr2=0x7f289c0b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cfea0 0x7f289c0b8480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f288400c400 tx=0x7f288400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 shutdown_connections 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f287c06adb0 0x7f287c06d270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f289c160920 0x7f289c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c0cfea0 0x7f289c0b8480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 --2- 172.21.15.64:0/2792605102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f289c0cf9f0 0x7f289c0b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 >> 172.21.15.64:0/2792605102 conn(0x7f289c0d3090 msgr2=0x7f289c06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 shutdown_connections 2023-11-24T06:27:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.694+0000 7f28a0d67640 1 -- 172.21.15.64:0/2792605102 wait complete. 2023-11-24T06:27:34.715 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-11-24T06:27:34.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:34.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.939+0000 7f52c994e640 1 Processor -- start 2023-11-24T06:27:34.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.939+0000 7f52c994e640 1 -- start start 2023-11-24T06:27:34.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c994e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c4143c40 0x7f52c4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c994e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f52c40717d0 0x7f52c40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c994e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52c40cca40 0x7f52c415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c994e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40723c0 con 0x7f52c40cca40 2023-11-24T06:27:34.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c994e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c4070e50 con 0x7f52c40717d0 2023-11-24T06:27:34.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c994e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40cce70 con 0x7f52c4143c40 2023-11-24T06:27:34.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c914d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52c40cca40 0x7f52c415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c914d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52c40cca40 0x7f52c415eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52926/0 (socket says 172.21.15.64:52926) 2023-11-24T06:27:34.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c914d640 1 -- 172.21.15.64:0/1646209738 learned_addr learned my addr 172.21.15.64:0/1646209738 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:34.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c894c640 1 --2- 172.21.15.64:0/1646209738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c4143c40 0x7f52c4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.941+0000 7f52c3fff640 1 --2- 172.21.15.64:0/1646209738 >> v2:172.21.15.162:3300/0 conn(0x7f52c40717d0 0x7f52c40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.942+0000 7f52c914d640 1 -- 172.21.15.64:0/1646209738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c4143c40 msgr2=0x7f52c4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.942+0000 7f52c914d640 1 --2- 172.21.15.64:0/1646209738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c4143c40 0x7f52c4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.942+0000 7f52c914d640 1 -- 172.21.15.64:0/1646209738 >> v2:172.21.15.162:3300/0 conn(0x7f52c40717d0 msgr2=0x7f52c40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.942+0000 7f52c914d640 1 --2- 172.21.15.64:0/1646209738 >> v2:172.21.15.162:3300/0 conn(0x7f52c40717d0 0x7f52c40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.942+0000 7f52c914d640 1 -- 172.21.15.64:0/1646209738 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4144020 con 0x7f52c40cca40 2023-11-24T06:27:34.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.942+0000 7f52c3fff640 1 --2- 172.21.15.64:0/1646209738 >> v2:172.21.15.162:3300/0 conn(0x7f52c40717d0 0x7f52c40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:34.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.943+0000 7f52c914d640 1 --2- 172.21.15.64:0/1646209738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c40cca40 0x7f52c415eab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52c406e440 tx=0x7f52bc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c67c74890fbbdf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.943+0000 7f52c37fe640 1 -- 172.21.15.64:0/1646209738 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc02c9f0 con 0x7f52c40cca40 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.943+0000 7f52c37fe640 1 -- 172.21.15.64:0/1646209738 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc02cb50 con 0x7f52c40cca40 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.943+0000 7f52c37fe640 1 -- 172.21.15.64:0/1646209738 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0343f0 con 0x7f52c40cca40 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 -- 172.21.15.64:0/1646209738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c40cca40 msgr2=0x7f52c415eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 --2- 172.21.15.64:0/1646209738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c40cca40 0x7f52c415eab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52c406e440 tx=0x7f52bc02be50 comp rx=0 tx=0).stop 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 -- 172.21.15.64:0/1646209738 shutdown_connections 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 --2- 172.21.15.64:0/1646209738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c40cca40 0x7f52c415eab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 --2- 172.21.15.64:0/1646209738 >> v2:172.21.15.162:3300/0 conn(0x7f52c40717d0 0x7f52c40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 --2- 172.21.15.64:0/1646209738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c4143c40 0x7f52c4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 -- 172.21.15.64:0/1646209738 >> 172.21.15.64:0/1646209738 conn(0x7f52c406a800 msgr2=0x7f52c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 -- 172.21.15.64:0/1646209738 shutdown_connections 2023-11-24T06:27:34.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.944+0000 7f52c994e640 1 -- 172.21.15.64:0/1646209738 wait complete. 2023-11-24T06:27:34.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.946+0000 7f52c994e640 1 Processor -- start 2023-11-24T06:27:34.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.946+0000 7f52c994e640 1 -- start start 2023-11-24T06:27:34.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c994e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c40717d0 0x7f52c415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c994e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40cca40 0x7f52c40ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c994e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4143c40 0x7f52c413b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c994e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40ab8c0 con 0x7f52c4143c40 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c994e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415aeb0 con 0x7f52c40cca40 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c994e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40aa5c0 con 0x7f52c40717d0 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c914d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4143c40 0x7f52c413b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c914d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4143c40 0x7f52c413b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52934/0 (socket says 172.21.15.64:52934) 2023-11-24T06:27:34.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c914d640 1 -- 172.21.15.64:0/2402320104 learned_addr learned my addr 172.21.15.64:0/2402320104 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.947+0000 7f52c894c640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c40717d0 0x7f52c415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.948+0000 7f52c3fff640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40cca40 0x7f52c40ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c894c640 1 -- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40cca40 msgr2=0x7f52c40ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c894c640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40cca40 0x7f52c40ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c894c640 1 -- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4143c40 msgr2=0x7f52c413b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c894c640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4143c40 0x7f52c413b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c894c640 1 -- 172.21.15.64:0/2402320104 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009b30 con 0x7f52c40717d0 2023-11-24T06:27:34.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c914d640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4143c40 0x7f52c413b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:34.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c3fff640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40cca40 0x7f52c40ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:34.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.949+0000 7f52c894c640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c40717d0 0x7f52c415a840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d620 tx=0x7f52b400dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.950+0000 7f52c17fa640 1 -- 172.21.15.64:0/2402320104 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400b620 con 0x7f52c40717d0 2023-11-24T06:27:34.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.950+0000 7f52c17fa640 1 -- 172.21.15.64:0/2402320104 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b400b780 con 0x7f52c40717d0 2023-11-24T06:27:34.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.950+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c40aa8a0 con 0x7f52c40717d0 2023-11-24T06:27:34.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.950+0000 7f52c17fa640 1 -- 172.21.15.64:0/2402320104 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400bb10 con 0x7f52c40717d0 2023-11-24T06:27:34.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.950+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c40aae60 con 0x7f52c40717d0 2023-11-24T06:27:34.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.951+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4067450 con 0x7f52c40717d0 2023-11-24T06:27:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.953+0000 7f52c17fa640 1 -- 172.21.15.64:0/2402320104 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f52b400bc70 con 0x7f52c40717d0 2023-11-24T06:27:34.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.954+0000 7f52c17fa640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f52b006adb0 0x7f52b006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.954+0000 7f52c17fa640 1 -- 172.21.15.64:0/2402320104 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f52b4089e80 con 0x7f52c40717d0 2023-11-24T06:27:34.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.954+0000 7f52c3fff640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f52b006adb0 0x7f52b006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.955+0000 7f52c3fff640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f52b006adb0 0x7f52b006d270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52c413fd60 tx=0x7f52b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:34.968+0000 7f52c17fa640 1 -- 172.21.15.64:0/2402320104 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52b4055590 con 0x7f52c40717d0 2023-11-24T06:27:35.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.203+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52c40715b0 con 0x7f52c40717d0 2023-11-24T06:27:35.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.204+0000 7f52c17fa640 1 -- 172.21.15.64:0/2402320104 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f52b4058bb0 con 0x7f52c40717d0 2023-11-24T06:27:35.207 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:35.207 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-24T06:27:35.207 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.206+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f52b006adb0 msgr2=0x7f52b006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.206+0000 7f52c994e640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f52b006adb0 0x7f52b006d270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52c413fd60 tx=0x7f52b8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c40717d0 msgr2=0x7f52c415a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c40717d0 0x7f52c415a840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d620 tx=0x7f52b400dae0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 shutdown_connections 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f52b006adb0 0x7f52b006d270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4143c40 0x7f52c413b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40cca40 0x7f52c40ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 --2- 172.21.15.64:0/2402320104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52c40717d0 0x7f52c415a840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 >> 172.21.15.64:0/2402320104 conn(0x7f52c406a800 msgr2=0x7f52c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 shutdown_connections 2023-11-24T06:27:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:35.207+0000 7f52c994e640 1 -- 172.21.15.64:0/2402320104 wait complete. 2023-11-24T06:27:37.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.421+0000 7f421ff1d640 1 Processor -- start 2023-11-24T06:27:37.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.421+0000 7f421ff1d640 1 -- start start 2023-11-24T06:27:37.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.422+0000 7f421ff1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180af810 0x7f421806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.423+0000 7f421ff1d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421815d720 con 0x7f42180b3ce0 2023-11-24T06:27:37.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.423+0000 7f421ff1d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42180b4b00 con 0x7f421815d340 2023-11-24T06:27:37.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.423+0000 7f421ff1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42180b4dd0 con 0x7f42180af810 2023-11-24T06:27:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.423+0000 7f421e4ba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f42180b3ce0 0x7f4218160ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49958/0 (socket says 172.21.15.64:49958) 2023-11-24T06:27:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.423+0000 7f421e4ba640 1 -- 172.21.15.64:0/1510958367 learned_addr learned my addr 172.21.15.64:0/1510958367 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.423+0000 7f421d4b8640 1 --2- 172.21.15.64:0/1510958367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180af810 0x7f421806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.424+0000 7f421ccb7640 1 -- 172.21.15.64:0/1510958367 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236943424 0 0) 0x7f421815d720 con 0x7f42180b3ce0 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.424+0000 7f421ccb7640 1 -- 172.21.15.64:0/1510958367 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4200003520 con 0x7f42180b3ce0 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.424+0000 7f421ccb7640 1 -- 172.21.15.64:0/1510958367 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6897955 0 0) 0x7f42180b4b00 con 0x7f421815d340 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.424+0000 7f421ccb7640 1 -- 172.21.15.64:0/1510958367 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f421815d720 con 0x7f421815d340 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.424+0000 7f421d4b8640 1 -- 172.21.15.64:0/1510958367 >> v1:172.21.15.162:6789/0 conn(0x7f421815d340 legacy=0x7f42180b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.424+0000 7f421d4b8640 1 -- 172.21.15.64:0/1510958367 >> v1:172.21.15.64:6789/0 conn(0x7f42180b3ce0 legacy=0x7f4218160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.424+0000 7f421d4b8640 1 -- 172.21.15.64:0/1510958367 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180b3200 con 0x7f42180af810 2023-11-24T06:27:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.426+0000 7f421d4b8640 1 --2- 172.21.15.64:0/1510958367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180af810 0x7f421806a3e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f420c0097d0 tx=0x7f420c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895c88850e9ddde3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.426+0000 7f421ccb7640 1 -- 172.21.15.64:0/1510958367 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c031770 con 0x7f42180af810 2023-11-24T06:27:37.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.426+0000 7f421ccb7640 1 -- 172.21.15.64:0/1510958367 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c0318d0 con 0x7f42180af810 2023-11-24T06:27:37.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.426+0000 7f421ccb7640 1 -- 172.21.15.64:0/1510958367 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c031c40 con 0x7f42180af810 2023-11-24T06:27:37.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.427+0000 7f421ff1d640 1 -- 172.21.15.64:0/1510958367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180af810 msgr2=0x7f421806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.427+0000 7f421ff1d640 1 --2- 172.21.15.64:0/1510958367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180af810 0x7f421806a3e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f420c0097d0 tx=0x7f420c02b810 comp rx=0 tx=0).stop 2023-11-24T06:27:37.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.427+0000 7f421ff1d640 1 -- 172.21.15.64:0/1510958367 shutdown_connections 2023-11-24T06:27:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.427+0000 7f421ff1d640 1 --2- 172.21.15.64:0/1510958367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180af810 0x7f421806a3e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.427+0000 7f421ff1d640 1 -- 172.21.15.64:0/1510958367 >> 172.21.15.64:0/1510958367 conn(0x7f42180b1dd0 msgr2=0x7f42180b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.427+0000 7f421ff1d640 1 -- 172.21.15.64:0/1510958367 shutdown_connections 2023-11-24T06:27:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.427+0000 7f421ff1d640 1 -- 172.21.15.64:0/1510958367 wait complete. 2023-11-24T06:27:37.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.429+0000 7f421ff1d640 1 Processor -- start 2023-11-24T06:27:37.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.429+0000 7f421ff1d640 1 -- start start 2023-11-24T06:27:37.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421ff1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180af810 0x7f421813afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421ff1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180b3ce0 0x7f42181532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421ff1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421815d340 0x7f421813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421ff1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421813c8e0 con 0x7f421815d340 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421ff1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421815e260 con 0x7f42180af810 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421ff1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421813c2d0 con 0x7f42180b3ce0 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421e4ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421815d340 0x7f421813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421e4ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421815d340 0x7f421813cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52962/0 (socket says 172.21.15.64:52962) 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421e4ba640 1 -- 172.21.15.64:0/2596534549 learned_addr learned my addr 172.21.15.64:0/2596534549 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421d4b8640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180b3ce0 0x7f42181532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421dcb9640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180af810 0x7f421813afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421e4ba640 1 -- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180b3ce0 msgr2=0x7f42181532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.430+0000 7f421e4ba640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180b3ce0 0x7f42181532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f421e4ba640 1 -- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180af810 msgr2=0x7f421813afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f421e4ba640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180af810 0x7f421813afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f421e4ba640 1 -- 172.21.15.64:0/2596534549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c009400 con 0x7f421815d340 2023-11-24T06:27:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f421dcb9640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180af810 0x7f421813afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f421e4ba640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421815d340 0x7f421813cd00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f421000c450 tx=0x7f421000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f42167fc640 1 -- 172.21.15.64:0/2596534549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210005ce0 con 0x7f421815d340 2023-11-24T06:27:37.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f42167fc640 1 -- 172.21.15.64:0/2596534549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4210005e40 con 0x7f421815d340 2023-11-24T06:27:37.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421815dca0 con 0x7f421815d340 2023-11-24T06:27:37.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f42167fc640 1 -- 172.21.15.64:0/2596534549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210007270 con 0x7f421815d340 2023-11-24T06:27:37.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.431+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421813c500 con 0x7f421815d340 2023-11-24T06:27:37.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.432+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42180b2af0 con 0x7f421815d340 2023-11-24T06:27:37.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.433+0000 7f42167fc640 1 -- 172.21.15.64:0/2596534549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f42100073d0 con 0x7f421815d340 2023-11-24T06:27:37.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.434+0000 7f42167fc640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f420406ade0 0x7f420406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.435+0000 7f42167fc640 1 -- 172.21.15.64:0/2596534549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4210088670 con 0x7f421815d340 2023-11-24T06:27:37.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.435+0000 7f421dcb9640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f420406ade0 0x7f420406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.435+0000 7f421dcb9640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f420406ade0 0x7f420406d2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4208007fd0 tx=0x7f4208009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.445+0000 7f42167fc640 1 -- 172.21.15.64:0/2596534549 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4210053c20 con 0x7f421815d340 2023-11-24T06:27:37.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.687+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f421813ca20 con 0x7f421815d340 2023-11-24T06:27:37.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.687+0000 7f42167fc640 1 -- 172.21.15.64:0/2596534549 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f421813ca20 con 0x7f421815d340 2023-11-24T06:27:37.690 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:37.690 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T06:27:37.691 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f420406ade0 msgr2=0x7f420406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f420406ade0 0x7f420406d2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4208007fd0 tx=0x7f4208009040 comp rx=0 tx=0).stop 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421815d340 msgr2=0x7f421813cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421815d340 0x7f421813cd00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f421000c450 tx=0x7f421000c910 comp rx=0 tx=0).stop 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 shutdown_connections 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f420406ade0 0x7f420406d2a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f421815d340 0x7f421813cd00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42180b3ce0 0x7f42181532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 --2- 172.21.15.64:0/2596534549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180af810 0x7f421813afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 >> 172.21.15.64:0/2596534549 conn(0x7f42180b1dd0 msgr2=0x7f421806ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:37.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 shutdown_connections 2023-11-24T06:27:37.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.690+0000 7f421ff1d640 1 -- 172.21.15.64:0/2596534549 wait complete. 2023-11-24T06:27:37.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:37.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.915+0000 7f27c931d640 1 Processor -- start 2023-11-24T06:27:37.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.916+0000 7f27c931d640 1 -- start start 2023-11-24T06:27:37.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.917+0000 7f27c931d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c40cd240 0x7f27c40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.917+0000 7f27c931d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f27c409a170 0x7f27c40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.917+0000 7f27c931d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c406cea0 con 0x7f27c406c6b0 2023-11-24T06:27:37.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.917+0000 7f27c931d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c406cbc0 con 0x7f27c409a170 2023-11-24T06:27:37.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.917+0000 7f27c931d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c406f350 con 0x7f27c40cd240 2023-11-24T06:27:37.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.918+0000 7f27c2ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c40cd240 0x7f27c40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.918+0000 7f27c2ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c40cd240 0x7f27c40b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:34580/0 (socket says 172.21.15.64:34580) 2023-11-24T06:27:37.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.918+0000 7f27c2ffd640 1 -- 172.21.15.64:0/4046594658 learned_addr learned my addr 172.21.15.64:0/4046594658 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:37.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.918+0000 7f27c37fe640 1 --2- 172.21.15.64:0/4046594658 >> v2:172.21.15.162:3300/0 conn(0x7f27c409a170 0x7f27c40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.919+0000 7f27c2ffd640 1 -- 172.21.15.64:0/4046594658 >> v2:172.21.15.162:3300/0 conn(0x7f27c409a170 msgr2=0x7f27c40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:37.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.919+0000 7f27c2ffd640 1 --2- 172.21.15.64:0/4046594658 >> v2:172.21.15.162:3300/0 conn(0x7f27c409a170 0x7f27c40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.919+0000 7f27c2ffd640 1 -- 172.21.15.64:0/4046594658 >> v1:172.21.15.162:6789/0 conn(0x7f27c406c6b0 legacy=0x7f27c4099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.919+0000 7f27c2ffd640 1 -- 172.21.15.64:0/4046594658 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c406f630 con 0x7f27c40cd240 2023-11-24T06:27:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.920+0000 7f27c2ffd640 1 --2- 172.21.15.64:0/4046594658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c40cd240 0x7f27c40b3820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009e50 tx=0x7f27b402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fbafab0da556c5d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.920+0000 7f27c27fc640 1 -- 172.21.15.64:0/4046594658 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b402cb00 con 0x7f27c40cd240 2023-11-24T06:27:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.920+0000 7f27c27fc640 1 -- 172.21.15.64:0/4046594658 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b402cc60 con 0x7f27c40cd240 2023-11-24T06:27:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.920+0000 7f27c27fc640 1 -- 172.21.15.64:0/4046594658 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b40343f0 con 0x7f27c40cd240 2023-11-24T06:27:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 -- 172.21.15.64:0/4046594658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c40cd240 msgr2=0x7f27c40b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 --2- 172.21.15.64:0/4046594658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c40cd240 0x7f27c40b3820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009e50 tx=0x7f27b402bf60 comp rx=0 tx=0).stop 2023-11-24T06:27:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 -- 172.21.15.64:0/4046594658 shutdown_connections 2023-11-24T06:27:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 --2- 172.21.15.64:0/4046594658 >> v2:172.21.15.162:3300/0 conn(0x7f27c409a170 0x7f27c40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 --2- 172.21.15.64:0/4046594658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c40cd240 0x7f27c40b3820 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 -- 172.21.15.64:0/4046594658 >> 172.21.15.64:0/4046594658 conn(0x7f27c40b2d00 msgr2=0x7f27c40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 -- 172.21.15.64:0/4046594658 shutdown_connections 2023-11-24T06:27:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.921+0000 7f27c931d640 1 -- 172.21.15.64:0/4046594658 wait complete. 2023-11-24T06:27:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.923+0000 7f27c931d640 1 Processor -- start 2023-11-24T06:27:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.923+0000 7f27c931d640 1 -- start start 2023-11-24T06:27:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c931d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c406c6b0 0x7f27c4151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c931d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c409a170 0x7f27c40b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c931d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c40cd240 0x7f27c40c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c931d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40b4a70 con 0x7f27c40cd240 2023-11-24T06:27:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c931d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c4151ac0 con 0x7f27c406c6b0 2023-11-24T06:27:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c931d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40b4cc0 con 0x7f27c409a170 2023-11-24T06:27:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c40cd240 0x7f27c40c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c40cd240 0x7f27c40c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52966/0 (socket says 172.21.15.64:52966) 2023-11-24T06:27:37.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.924+0000 7f27c37fe640 1 -- 172.21.15.64:0/2852650721 learned_addr learned my addr 172.21.15.64:0/2852650721 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27ba7fc640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c409a170 0x7f27c40b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c2ffd640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c406c6b0 0x7f27c4151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c37fe640 1 -- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c409a170 msgr2=0x7f27c40b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c37fe640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c409a170 0x7f27c40b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c37fe640 1 -- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c406c6b0 msgr2=0x7f27c4151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c37fe640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c406c6b0 0x7f27c4151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c37fe640 1 -- 172.21.15.64:0/2852650721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b4009b30 con 0x7f27c40cd240 2023-11-24T06:27:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c2ffd640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c406c6b0 0x7f27c4151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.925+0000 7f27c37fe640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c40cd240 0x7f27c40c6b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e7f0 tx=0x7f27bc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.926+0000 7f27bbfff640 1 -- 172.21.15.64:0/2852650721 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc009510 con 0x7f27c40cd240 2023-11-24T06:27:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.926+0000 7f27bbfff640 1 -- 172.21.15.64:0/2852650721 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc00ee40 con 0x7f27c40cd240 2023-11-24T06:27:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.926+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c40b4fa0 con 0x7f27c40cd240 2023-11-24T06:27:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.926+0000 7f27bbfff640 1 -- 172.21.15.64:0/2852650721 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00cb80 con 0x7f27c40cd240 2023-11-24T06:27:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.926+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c40b5560 con 0x7f27c40cd240 2023-11-24T06:27:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.927+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c40b5780 con 0x7f27c40cd240 2023-11-24T06:27:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.928+0000 7f27bbfff640 1 -- 172.21.15.64:0/2852650721 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f27bc00cce0 con 0x7f27c40cd240 2023-11-24T06:27:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.929+0000 7f27bbfff640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f27a406adb0 0x7f27a406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.929+0000 7f27bbfff640 1 -- 172.21.15.64:0/2852650721 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f27bc089280 con 0x7f27c40cd240 2023-11-24T06:27:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.929+0000 7f27c2ffd640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f27a406adb0 0x7f27a406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.930+0000 7f27c2ffd640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f27a406adb0 0x7f27a406d270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f27b4035000 tx=0x7f27b4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:37.933+0000 7f27bbfff640 1 -- 172.21.15.64:0/2852650721 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27bc054750 con 0x7f27c40cd240 2023-11-24T06:27:38.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.133+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27c406ef10 con 0x7f27c40cd240 2023-11-24T06:27:38.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.134+0000 7f27bbfff640 1 -- 172.21.15.64:0/2852650721 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f27bc057d70 con 0x7f27c40cd240 2023-11-24T06:27:38.137 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:38.137 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":18,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:37.873678+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f27a406adb0 msgr2=0x7f27a406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f27a406adb0 0x7f27a406d270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f27b4035000 tx=0x7f27b4030000 comp rx=0 tx=0).stop 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c40cd240 msgr2=0x7f27c40c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c40cd240 0x7f27c40c6b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e7f0 tx=0x7f27bc00ebf0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 shutdown_connections 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f27a406adb0 0x7f27a406d270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c40cd240 0x7f27c40c6b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c409a170 0x7f27c40b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 --2- 172.21.15.64:0/2852650721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c406c6b0 0x7f27c4151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.137+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 >> 172.21.15.64:0/2852650721 conn(0x7f27c40b2d00 msgr2=0x7f27c40cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.138+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 shutdown_connections 2023-11-24T06:27:38.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.138+0000 7f27c931d640 1 -- 172.21.15.64:0/2852650721 wait complete. 2023-11-24T06:27:38.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:38.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:38.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.348+0000 7fd7f842d640 1 Processor -- start 2023-11-24T06:27:38.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.348+0000 7fd7f842d640 1 -- start start 2023-11-24T06:27:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.349+0000 7fd7f842d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f00ad880 0x7fd7f00717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.349+0000 7fd7f842d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd7f006bb00 0x7fd7f00b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.349+0000 7fd7f842d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f015e680 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.349+0000 7fd7f842d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f0071cf0 con 0x7fd7f006bb00 2023-11-24T06:27:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f842d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f015dd50 con 0x7fd7f00ad880 2023-11-24T06:27:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f69ca640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd7f006bb00 0x7fd7f00b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f69ca640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd7f006bb00 0x7fd7f00b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52976/0 (socket says 172.21.15.64:52976) 2023-11-24T06:27:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f69ca640 1 -- 172.21.15.64:0/3620247192 learned_addr learned my addr 172.21.15.64:0/3620247192 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f61c9640 1 --2- 172.21.15.64:0/3620247192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f00ad880 0x7fd7f00717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f69ca640 1 -- 172.21.15.64:0/3620247192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f00ad880 msgr2=0x7fd7f00717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f69ca640 1 --2- 172.21.15.64:0/3620247192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f00ad880 0x7fd7f00717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f69ca640 1 -- 172.21.15.64:0/3620247192 >> v1:172.21.15.64:6789/0 conn(0x7fd7f00ac8d0 legacy=0x7fd7f006b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f51c7640 1 -- 172.21.15.64:0/3620247192 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423112423 0 0) 0x7fd7f015e680 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.350+0000 7fd7f69ca640 1 -- 172.21.15.64:0/3620247192 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f015e030 con 0x7fd7f006bb00 2023-11-24T06:27:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.352+0000 7fd7f69ca640 1 --2- 172.21.15.64:0/3620247192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f006bb00 0x7fd7f00b26a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e50 tx=0x7fd7e802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9377a9b4dc711021 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.352+0000 7fd7f51c7640 1 -- 172.21.15.64:0/3620247192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e802cb90 con 0x7fd7f006bb00 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.352+0000 7fd7f51c7640 1 -- 172.21.15.64:0/3620247192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e802ccf0 con 0x7fd7f006bb00 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.352+0000 7fd7f51c7640 1 -- 172.21.15.64:0/3620247192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8034610 con 0x7fd7f006bb00 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.352+0000 7fd7f842d640 1 -- 172.21.15.64:0/3620247192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f006bb00 msgr2=0x7fd7f00b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.352+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3620247192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f006bb00 0x7fd7f00b26a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e50 tx=0x7fd7e802bff0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.353+0000 7fd7f842d640 1 -- 172.21.15.64:0/3620247192 shutdown_connections 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.353+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3620247192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f006bb00 0x7fd7f00b26a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.353+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3620247192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f00ad880 0x7fd7f00717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.353+0000 7fd7f842d640 1 -- 172.21.15.64:0/3620247192 >> 172.21.15.64:0/3620247192 conn(0x7fd7f0070a80 msgr2=0x7fd7f0070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.353+0000 7fd7f842d640 1 -- 172.21.15.64:0/3620247192 shutdown_connections 2023-11-24T06:27:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.353+0000 7fd7f842d640 1 -- 172.21.15.64:0/3620247192 wait complete. 2023-11-24T06:27:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.355+0000 7fd7f842d640 1 Processor -- start 2023-11-24T06:27:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.355+0000 7fd7f842d640 1 -- start start 2023-11-24T06:27:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f842d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f006bb00 0x7fd7f014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f842d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f00ac8d0 0x7fd7f006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f842d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f00ad880 0x7fd7f00b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f842d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f006e530 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f842d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01f6e80 con 0x7fd7f00ad880 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f842d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01f6100 con 0x7fd7f006bb00 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f59c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f00ac8d0 0x7fd7f006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f59c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f00ac8d0 0x7fd7f006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52980/0 (socket says 172.21.15.64:52980) 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f59c8640 1 -- 172.21.15.64:0/3013332704 learned_addr learned my addr 172.21.15.64:0/3013332704 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f61c9640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f006bb00 0x7fd7f014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f59c8640 1 -- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f006bb00 msgr2=0x7fd7f014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f69ca640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f00ad880 0x7fd7f00b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f59c8640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f006bb00 0x7fd7f014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f59c8640 1 -- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f00ad880 msgr2=0x7fd7f00b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.356+0000 7fd7f59c8640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f00ad880 0x7fd7f00b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.357+0000 7fd7f59c8640 1 -- 172.21.15.64:0/3013332704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8009b30 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.357+0000 7fd7f59c8640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f00ac8d0 0x7fd7f006dce0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c4b0 tx=0x7fd7e000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.357+0000 7fd7e6ffd640 1 -- 172.21.15.64:0/3013332704 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0005480 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.357+0000 7fd7e6ffd640 1 -- 172.21.15.64:0/3013332704 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e00035f0 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.357+0000 7fd7e6ffd640 1 -- 172.21.15.64:0/3013332704 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00039e0 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.357+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f014f530 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.357+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f01f6680 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.358+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f00675d0 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.359+0000 7fd7e6ffd640 1 -- 172.21.15.64:0/3013332704 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fd7e0003b40 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.360+0000 7fd7e6ffd640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd7d406adb0 0x7fd7d406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.360+0000 7fd7e6ffd640 1 -- 172.21.15.64:0/3013332704 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7e0088830 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.361+0000 7fd7f61c9640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd7d406adb0 0x7fd7d406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.361+0000 7fd7f61c9640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd7d406adb0 0x7fd7d406d270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec006fd0 tx=0x7fd7ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.365+0000 7fd7e6ffd640 1 -- 172.21.15.64:0/3013332704 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7e0053c70 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.559+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7f006b640 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.560+0000 7fd7e6ffd640 1 -- 172.21.15.64:0/3013332704 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7e0017070 con 0x7fd7f00ac8d0 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd7d406adb0 msgr2=0x7fd7d406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd7d406adb0 0x7fd7d406d270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec006fd0 tx=0x7fd7ec008040 comp rx=0 tx=0).stop 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f00ac8d0 msgr2=0x7fd7f006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f00ac8d0 0x7fd7f006dce0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c4b0 tx=0x7fd7e000c970 comp rx=0 tx=0).stop 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 shutdown_connections 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd7d406adb0 0x7fd7d406d270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f00ad880 0x7fd7f00b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f00ac8d0 0x7fd7f006dce0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 --2- 172.21.15.64:0/3013332704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7f006bb00 0x7fd7f014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 >> 172.21.15.64:0/3013332704 conn(0x7fd7f0070a80 msgr2=0x7fd7f01558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 shutdown_connections 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.563+0000 7fd7f842d640 1 -- 172.21.15.64:0/3013332704 wait complete. 2023-11-24T06:27:38.584 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:27:38.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.805+0000 7f2368f28640 1 Processor -- start 2023-11-24T06:27:38.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.805+0000 7f2368f28640 1 -- start start 2023-11-24T06:27:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f2368f28640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2364143c40 0x7f2364071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f2368f28640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23640717d0 0x7f23640cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f2368f28640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2364070e50 con 0x7f23640cca40 2023-11-24T06:27:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f2368f28640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23640723c0 con 0x7f23640717d0 2023-11-24T06:27:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f2368f28640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23640cce40 con 0x7f2364143c40 2023-11-24T06:27:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f235affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23640717d0 0x7f23640cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f235affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23640717d0 0x7f23640cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52996/0 (socket says 172.21.15.64:52996) 2023-11-24T06:27:38.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.807+0000 7f235affd640 1 -- 172.21.15.64:0/2762036884 learned_addr learned my addr 172.21.15.64:0/2762036884 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:38.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.808+0000 7f23637fe640 1 --2- 172.21.15.64:0/2762036884 >> v2:172.21.15.162:3300/0 conn(0x7f2364143c40 0x7f2364071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.808+0000 7f235affd640 1 -- 172.21.15.64:0/2762036884 >> v2:172.21.15.162:3300/0 conn(0x7f2364143c40 msgr2=0x7f2364071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.808+0000 7f235affd640 1 --2- 172.21.15.64:0/2762036884 >> v2:172.21.15.162:3300/0 conn(0x7f2364143c40 0x7f2364071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.808+0000 7f235affd640 1 -- 172.21.15.64:0/2762036884 >> v1:172.21.15.162:6789/0 conn(0x7f23640cca40 legacy=0x7f236415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.808+0000 7f235affd640 1 -- 172.21.15.64:0/2762036884 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236406e170 con 0x7f23640717d0 2023-11-24T06:27:38.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.809+0000 7f235affd640 1 --2- 172.21.15.64:0/2762036884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23640717d0 0x7f23640cc500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f236406e410 tx=0x7f234c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296dc0c723603c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.809+0000 7f2362ffd640 1 -- 172.21.15.64:0/2762036884 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c031870 con 0x7f23640717d0 2023-11-24T06:27:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.809+0000 7f2362ffd640 1 -- 172.21.15.64:0/2762036884 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c0319d0 con 0x7f23640717d0 2023-11-24T06:27:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.809+0000 7f2362ffd640 1 -- 172.21.15.64:0/2762036884 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c031d60 con 0x7f23640717d0 2023-11-24T06:27:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 -- 172.21.15.64:0/2762036884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23640717d0 msgr2=0x7f23640cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 --2- 172.21.15.64:0/2762036884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23640717d0 0x7f23640cc500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f236406e410 tx=0x7f234c02b910 comp rx=0 tx=0).stop 2023-11-24T06:27:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 -- 172.21.15.64:0/2762036884 shutdown_connections 2023-11-24T06:27:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 --2- 172.21.15.64:0/2762036884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23640717d0 0x7f23640cc500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 --2- 172.21.15.64:0/2762036884 >> v2:172.21.15.162:3300/0 conn(0x7f2364143c40 0x7f2364071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 -- 172.21.15.64:0/2762036884 >> 172.21.15.64:0/2762036884 conn(0x7f236406a800 msgr2=0x7f236406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 -- 172.21.15.64:0/2762036884 shutdown_connections 2023-11-24T06:27:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.810+0000 7f2368f28640 1 -- 172.21.15.64:0/2762036884 wait complete. 2023-11-24T06:27:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.812+0000 7f2368f28640 1 Processor -- start 2023-11-24T06:27:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.812+0000 7f2368f28640 1 -- start start 2023-11-24T06:27:38.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2368f28640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23640717d0 0x7f236413b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2368f28640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23640cca40 0x7f236415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2368f28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2364143c40 0x7f236413a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2368f28640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236413c9c0 con 0x7f2364143c40 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2368f28640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23640cae30 con 0x7f23640cca40 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2368f28640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236413bdb0 con 0x7f23640717d0 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2363fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2364143c40 0x7f236413a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2363fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2364143c40 0x7f236413a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52998/0 (socket says 172.21.15.64:52998) 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.813+0000 7f2363fff640 1 -- 172.21.15.64:0/3414124140 learned_addr learned my addr 172.21.15.64:0/3414124140 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f23637fe640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23640717d0 0x7f236413b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f235affd640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23640cca40 0x7f236415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2363fff640 1 -- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23640717d0 msgr2=0x7f236413b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2363fff640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23640717d0 0x7f236413b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2363fff640 1 -- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23640cca40 msgr2=0x7f236415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2363fff640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23640cca40 0x7f236415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2363fff640 1 -- 172.21.15.64:0/3414124140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0094d0 con 0x7f2364143c40 2023-11-24T06:27:38.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f235affd640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23640cca40 0x7f236415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:38.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2363fff640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2364143c40 0x7f236413a820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f235c001eb0 tx=0x7f235c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2360ff9640 1 -- 172.21.15.64:0/3414124140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0052d0 con 0x7f2364143c40 2023-11-24T06:27:38.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2360ff9640 1 -- 172.21.15.64:0/3414124140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c005430 con 0x7f2364143c40 2023-11-24T06:27:38.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2360ff9640 1 -- 172.21.15.64:0/3414124140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c003810 con 0x7f2364143c40 2023-11-24T06:27:38.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.814+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236413c090 con 0x7f2364143c40 2023-11-24T06:27:38.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.815+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236413c650 con 0x7f2364143c40 2023-11-24T06:27:38.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.816+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236413cb30 con 0x7f2364143c40 2023-11-24T06:27:38.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.817+0000 7f2360ff9640 1 -- 172.21.15.64:0/3414124140 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f235c003970 con 0x7f2364143c40 2023-11-24T06:27:38.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.817+0000 7f2360ff9640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f234406adb0 0x7f234406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.818+0000 7f2360ff9640 1 -- 172.21.15.64:0/3414124140 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f235c088740 con 0x7f2364143c40 2023-11-24T06:27:38.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.818+0000 7f23637fe640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f234406adb0 0x7f234406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.818+0000 7f23637fe640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f234406adb0 0x7f234406d270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2354003aa0 tx=0x7f2354009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:38.822+0000 7f2360ff9640 1 -- 172.21.15.64:0/3414124140 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f235c053c30 con 0x7f2364143c40 2023-11-24T06:27:39.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.027+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2364067450 con 0x7f2364143c40 2023-11-24T06:27:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.027+0000 7f2360ff9640 1 -- 172.21.15.64:0/3414124140 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f235c017090 con 0x7f2364143c40 2023-11-24T06:27:39.030 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:39.030 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T06:27:39.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:27:39.030 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T06:27:39.031 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T06:27:39.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:27:39.031 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T06:27:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f234406adb0 msgr2=0x7f234406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f234406adb0 0x7f234406d270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2354003aa0 tx=0x7f2354009830 comp rx=0 tx=0).stop 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2364143c40 msgr2=0x7f236413a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2364143c40 0x7f236413a820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f235c001eb0 tx=0x7f235c00c890 comp rx=0 tx=0).stop 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 shutdown_connections 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f234406adb0 0x7f234406d270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2364143c40 0x7f236413a820 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23640cca40 0x7f236415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 --2- 172.21.15.64:0/3414124140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23640717d0 0x7f236413b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 >> 172.21.15.64:0/3414124140 conn(0x7f236406a800 msgr2=0x7f23640aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 shutdown_connections 2023-11-24T06:27:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.030+0000 7f2368f28640 1 -- 172.21.15.64:0/3414124140 wait complete. 2023-11-24T06:27:39.051 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:27:39.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:27:39.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.281+0000 7f5f07fff640 1 Processor -- start 2023-11-24T06:27:39.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.281+0000 7f5f07fff640 1 -- start start 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f07fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f08143c40 0x7f5f08071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f0806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f07fff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f08070e50 con 0x7f5f080717d0 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f07fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f080723c0 con 0x7f5f08143c40 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f07fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08144020 con 0x7f5f080ccc10 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f06ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f08143c40 0x7f5f08071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f06ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f08143c40 0x7f5f08071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53000/0 (socket says 172.21.15.64:53000) 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f06ffd640 1 -- 172.21.15.64:0/2238214963 learned_addr learned my addr 172.21.15.64:0/2238214963 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:39.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.283+0000 7f5f077fe640 1 --2- 172.21.15.64:0/2238214963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f0806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.284+0000 7f5f06ffd640 1 -- 172.21.15.64:0/2238214963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 msgr2=0x7f5f0806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.284+0000 7f5f06ffd640 1 --2- 172.21.15.64:0/2238214963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f0806d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.284+0000 7f5f06ffd640 1 -- 172.21.15.64:0/2238214963 >> v1:172.21.15.162:6789/0 conn(0x7f5f080717d0 legacy=0x7f5f080cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.284+0000 7f5f06ffd640 1 -- 172.21.15.64:0/2238214963 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f08071e00 con 0x7f5f08143c40 2023-11-24T06:27:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.285+0000 7f5f06ffd640 1 --2- 172.21.15.64:0/2238214963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f08143c40 0x7f5f08071290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009870 tx=0x7f5ef802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831fb29b1f9143d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.285+0000 7f5f05ffb640 1 -- 172.21.15.64:0/2238214963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8031640 con 0x7f5f08143c40 2023-11-24T06:27:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f05ffb640 1 -- 172.21.15.64:0/2238214963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef80317a0 con 0x7f5f08143c40 2023-11-24T06:27:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f05ffb640 1 -- 172.21.15.64:0/2238214963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8031b30 con 0x7f5f08143c40 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 -- 172.21.15.64:0/2238214963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f08143c40 msgr2=0x7f5f08071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 --2- 172.21.15.64:0/2238214963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f08143c40 0x7f5f08071290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009870 tx=0x7f5ef802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 -- 172.21.15.64:0/2238214963 shutdown_connections 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 --2- 172.21.15.64:0/2238214963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f0806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 --2- 172.21.15.64:0/2238214963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f08143c40 0x7f5f08071290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 -- 172.21.15.64:0/2238214963 >> 172.21.15.64:0/2238214963 conn(0x7f5f0806a800 msgr2=0x7f5f0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 -- 172.21.15.64:0/2238214963 shutdown_connections 2023-11-24T06:27:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.286+0000 7f5f07fff640 1 -- 172.21.15.64:0/2238214963 wait complete. 2023-11-24T06:27:39.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.288+0000 7f5f07fff640 1 Processor -- start 2023-11-24T06:27:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.288+0000 7f5f07fff640 1 -- start start 2023-11-24T06:27:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.289+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f080717d0 0x7f5f080b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.289+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f08139d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.289+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f08143c40 0x7f5f08141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.289+0000 7f5f07fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08139920 con 0x7f5f080717d0 2023-11-24T06:27:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.289+0000 7f5f07fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08138ca0 con 0x7f5f08143c40 2023-11-24T06:27:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.289+0000 7f5f07fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08138f80 con 0x7f5f080ccc10 2023-11-24T06:27:39.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.289+0000 7f5f06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f080717d0 0x7f5f080b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f080717d0 0x7f5f080b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53002/0 (socket says 172.21.15.64:53002) 2023-11-24T06:27:39.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 -- 172.21.15.64:0/1809727188 learned_addr learned my addr 172.21.15.64:0/1809727188 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:39.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f067fc640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f08139d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f077fe640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f08143c40 0x7f5f08141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 -- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 msgr2=0x7f5f08139d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f08139d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 -- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f08143c40 msgr2=0x7f5f08141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f08143c40 0x7f5f08141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 -- 172.21.15.64:0/1809727188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef80094d0 con 0x7f5f080717d0 2023-11-24T06:27:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.290+0000 7f5f06ffd640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f080717d0 0x7f5f080b3830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8000c00 tx=0x7f5ef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.291+0000 7f5eef7fe640 1 -- 172.21.15.64:0/1809727188 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef803a930 con 0x7f5f080717d0 2023-11-24T06:27:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.291+0000 7f5eef7fe640 1 -- 172.21.15.64:0/1809727188 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef803aa90 con 0x7f5f080717d0 2023-11-24T06:27:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.291+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f08139260 con 0x7f5f080717d0 2023-11-24T06:27:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.291+0000 7f5eef7fe640 1 -- 172.21.15.64:0/1809727188 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef803ae40 con 0x7f5f080717d0 2023-11-24T06:27:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.291+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f080b3d70 con 0x7f5f080717d0 2023-11-24T06:27:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.292+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f080b3f60 con 0x7f5f080717d0 2023-11-24T06:27:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.293+0000 7f5eef7fe640 1 -- 172.21.15.64:0/1809727188 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f5ef802f050 con 0x7f5f080717d0 2023-11-24T06:27:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.294+0000 7f5eef7fe640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5ef406adb0 0x7f5ef406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.294+0000 7f5eef7fe640 1 -- 172.21.15.64:0/1809727188 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ef80aafd0 con 0x7f5f080717d0 2023-11-24T06:27:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.294+0000 7f5f067fc640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5ef406adb0 0x7f5ef406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.295+0000 7f5f067fc640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5ef406adb0 0x7f5ef406d270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f080a8830 tx=0x7f5efc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.298+0000 7f5eef7fe640 1 -- 172.21.15.64:0/1809727188 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ef8076410 con 0x7f5f080717d0 2023-11-24T06:27:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:39.493+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5f08139b90 con 0x7f5f080717d0 2023-11-24T06:27:40.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.327+0000 7f5eef7fe640 1 -- 172.21.15.64:0/1809727188 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5ef8079a30 con 0x7f5f080717d0 2023-11-24T06:27:40.330 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T06:27:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.331+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5ef406adb0 msgr2=0x7f5ef406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.331+0000 7f5f07fff640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5ef406adb0 0x7f5ef406d270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f080a8830 tx=0x7f5efc009040 comp rx=0 tx=0).stop 2023-11-24T06:27:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.331+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f080717d0 msgr2=0x7f5f080b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.331+0000 7f5f07fff640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f080717d0 0x7f5f080b3830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8000c00 tx=0x7f5ef80046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 shutdown_connections 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5ef406adb0 0x7f5ef406d270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f08143c40 0x7f5f08141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f080ccc10 0x7f5f08139d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 --2- 172.21.15.64:0/1809727188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f080717d0 0x7f5f080b3830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 >> 172.21.15.64:0/1809727188 conn(0x7f5f0806a800 msgr2=0x7f5f080cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 shutdown_connections 2023-11-24T06:27:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.332+0000 7f5f07fff640 1 -- 172.21.15.64:0/1809727188 wait complete. 2023-11-24T06:27:40.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:27:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.562+0000 7f052ffff640 1 Processor -- start 2023-11-24T06:27:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.563+0000 7f052ffff640 1 -- start start 2023-11-24T06:27:40.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.564+0000 7f052ffff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0530146770 0x7f053006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.564+0000 7f052ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300b2e10 con 0x7f0530071410 2023-11-24T06:27:40.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.564+0000 7f052ffff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0530146bc0 con 0x7f0530071f00 2023-11-24T06:27:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.564+0000 7f052ffff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053006cec0 con 0x7f0530146770 2023-11-24T06:27:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.565+0000 7f052effd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0530071410 0x7f05300717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49986/0 (socket says 172.21.15.64:49986) 2023-11-24T06:27:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.565+0000 7f052effd640 1 -- 172.21.15.64:0/1125485831 learned_addr learned my addr 172.21.15.64:0/1125485831 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.565+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022286567 0 0) 0x7f05300b2e10 con 0x7f0530071410 2023-11-24T06:27:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.565+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0518003520 con 0x7f0530071410 2023-11-24T06:27:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.565+0000 7f052f7fe640 1 --2- 172.21.15.64:0/1125485831 >> v2:172.21.15.162:3300/0 conn(0x7f0530146770 0x7f053006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.565+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2058483972 0 0) 0x7f0518003520 con 0x7f0530071410 2023-11-24T06:27:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 >> v2:172.21.15.162:3300/0 conn(0x7f0530146770 msgr2=0x7f053006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 --2- 172.21.15.64:0/1125485831 >> v2:172.21.15.162:3300/0 conn(0x7f0530146770 0x7f053006c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 >> v1:172.21.15.162:6789/0 conn(0x7f0530071f00 legacy=0x7f0530164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053006d1a0 con 0x7f0530071410 2023-11-24T06:27:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f05200019f0 con 0x7f0530071410 2023-11-24T06:27:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 >> v1:172.21.15.64:6789/0 conn(0x7f0530071410 legacy=0x7f05300717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f0518003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05180047c0 0x7f0518004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05180052a0 0x7f051800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053006cec0 con 0x7f05180039c0 2023-11-24T06:27:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051800c1c0 con 0x7f05180047c0 2023-11-24T06:27:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051800c380 con 0x7f05180052a0 2023-11-24T06:27:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052effd640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f0518003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052effd640 1 -- 172.21.15.64:0/1125485831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05180052a0 msgr2=0x7f051800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052effd640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05180052a0 0x7f051800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052effd640 1 -- 172.21.15.64:0/1125485831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05180047c0 msgr2=0x7f0518004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052effd640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05180047c0 0x7f0518004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052effd640 1 -- 172.21.15.64:0/1125485831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518003520 con 0x7f05180039c0 2023-11-24T06:27:40.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.566+0000 7f052f7fe640 1 -- 172.21.15.64:0/1125485831 reap_dead start 2023-11-24T06:27:40.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.567+0000 7f052effd640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f0518003de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f052000a760 tx=0x7f05180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1531d2f19f58e5b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.568+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520034630 con 0x7f05180039c0 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.568+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052002f070 con 0x7f05180039c0 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.568+0000 7f052dffb640 1 -- 172.21.15.64:0/1125485831 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520034a50 con 0x7f05180039c0 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.568+0000 7f052ffff640 1 -- 172.21.15.64:0/1125485831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 msgr2=0x7f0518003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.568+0000 7f052ffff640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f0518003de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f052000a760 tx=0x7f05180057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.568+0000 7f052ffff640 1 -- 172.21.15.64:0/1125485831 shutdown_connections 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.568+0000 7f052ffff640 1 --2- 172.21.15.64:0/1125485831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f0518003de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.569+0000 7f052ffff640 1 -- 172.21.15.64:0/1125485831 >> 172.21.15.64:0/1125485831 conn(0x7f053006a800 msgr2=0x7f053006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.569+0000 7f052ffff640 1 -- 172.21.15.64:0/1125485831 shutdown_connections 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.569+0000 7f052ffff640 1 -- 172.21.15.64:0/1125485831 wait complete. 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.570+0000 7f052ffff640 1 Processor -- start 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.571+0000 7f052ffff640 1 -- start start 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f05301443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05301331c0 0x7f0530133640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05301f66d0 0x7f05301f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05301347c0 con 0x7f05180039c0 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052ffff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530134c50 con 0x7f05301331c0 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052ffff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0530133e90 con 0x7f05301f66d0 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f05301443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f05301443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53020/0 (socket says 172.21.15.64:53020) 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 -- 172.21.15.64:0/816527419 learned_addr learned my addr 172.21.15.64:0/816527419 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 -- 172.21.15.64:0/816527419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05301f66d0 msgr2=0x7f05301f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05301f66d0 0x7f05301f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 -- 172.21.15.64:0/816527419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05301331c0 msgr2=0x7f0530133640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05301331c0 0x7f0530133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.572+0000 7f052effd640 1 -- 172.21.15.64:0/816527419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530134170 con 0x7f05180039c0 2023-11-24T06:27:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.573+0000 7f052effd640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f05301443e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f052002cab0 tx=0x7f0520001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.573+0000 7f05177fe640 1 -- 172.21.15.64:0/816527419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520004d60 con 0x7f05180039c0 2023-11-24T06:27:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.573+0000 7f05177fe640 1 -- 172.21.15.64:0/816527419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052003f470 con 0x7f05180039c0 2023-11-24T06:27:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.573+0000 7f05177fe640 1 -- 172.21.15.64:0/816527419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052003f800 con 0x7f05180039c0 2023-11-24T06:27:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.573+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0530134450 con 0x7f05180039c0 2023-11-24T06:27:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.573+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05301f7090 con 0x7f05180039c0 2023-11-24T06:27:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.574+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0530067450 con 0x7f05180039c0 2023-11-24T06:27:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.575+0000 7f05177fe640 1 -- 172.21.15.64:0/816527419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0520031030 con 0x7f05180039c0 2023-11-24T06:27:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.577+0000 7f05177fe640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f051c06ae70 0x7f051c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.577+0000 7f05177fe640 1 -- 172.21.15.64:0/816527419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f05200aa9a0 con 0x7f05180039c0 2023-11-24T06:27:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.577+0000 7f052e7fc640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f051c06ae70 0x7f051c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.577+0000 7f052e7fc640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f051c06ae70 0x7f051c06d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0530160be0 tx=0x7f05240079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.582+0000 7f05177fe640 1 -- 172.21.15.64:0/816527419 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0520075c30 con 0x7f05180039c0 2023-11-24T06:27:40.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:40.777+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0530133fd0 con 0x7f05180039c0 2023-11-24T06:27:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.341+0000 7f05177fe640 1 -- 172.21.15.64:0/816527419 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0530133fd0 con 0x7f05180039c0 2023-11-24T06:27:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T06:27:41.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f051c06ae70 msgr2=0x7f051c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f051c06ae70 0x7f051c06d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0530160be0 tx=0x7f05240079d0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 msgr2=0x7f05301443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f05301443e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f052002cab0 tx=0x7f0520001cb0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 shutdown_connections 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f051c06ae70 0x7f051c06d330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05301f66d0 0x7f05301f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05301331c0 0x7f0530133640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 --2- 172.21.15.64:0/816527419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180039c0 0x7f05301443e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.345+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 >> 172.21.15.64:0/816527419 conn(0x7f053006a800 msgr2=0x7f05300af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:41.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.346+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 shutdown_connections 2023-11-24T06:27:41.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.346+0000 7f052ffff640 1 -- 172.21.15.64:0/816527419 wait complete. 2023-11-24T06:27:41.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:27:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.574+0000 7f7915a59640 1 Processor -- start 2023-11-24T06:27:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.574+0000 7f7915a59640 1 -- start start 2023-11-24T06:27:41.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.575+0000 7f7915a59640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 0x7f791015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f7915a59640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791015f4e0 con 0x7f79100ccb50 2023-11-24T06:27:41.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f7915a59640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7910070ee0 con 0x7f79100adf20 2023-11-24T06:27:41.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f7915a59640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791015c450 con 0x7f7910070a30 2023-11-24T06:27:41.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f790ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f79100ccb50 0x7f7910143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49988/0 (socket says 172.21.15.64:49988) 2023-11-24T06:27:41.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f790ffff640 1 -- 172.21.15.64:0/4136936560 learned_addr learned my addr 172.21.15.64:0/4136936560 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:41.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f7915258640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 0x7f791015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586811108 0 0) 0x7f791015f4e0 con 0x7f79100ccb50 2023-11-24T06:27:41.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.576+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7900003520 con 0x7f79100ccb50 2023-11-24T06:27:41.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162528477 0 0) 0x7f7900003520 con 0x7f79100ccb50 2023-11-24T06:27:41.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 msgr2=0x7f791015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:41.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 0x7f791015bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 >> v1:172.21.15.162:6789/0 conn(0x7f79100adf20 legacy=0x7f79100cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910071130 con 0x7f79100ccb50 2023-11-24T06:27:41.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f78fc001670 con 0x7f79100ccb50 2023-11-24T06:27:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 >> v1:172.21.15.64:6789/0 conn(0x7f79100ccb50 legacy=0x7f7910143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 0x7f79000039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900003f20 0x7f79000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900004d00 0x7f7900005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791015c450 con 0x7f7900003f20 2023-11-24T06:27:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790000bb10 con 0x7f7900004d00 2023-11-24T06:27:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790000bcd0 con 0x7f7910070a30 2023-11-24T06:27:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f790ffff640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900003f20 0x7f79000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f7914a57640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 0x7f79000039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.577+0000 7f7915258640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900004d00 0x7f7900005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.578+0000 7f790ffff640 1 -- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 msgr2=0x7f79000039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.578+0000 7f790ffff640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7910070a30 0x7f79000039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.578+0000 7f790ffff640 1 -- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900004d00 msgr2=0x7f7900005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.578+0000 7f790ffff640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900004d00 0x7f7900005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.578+0000 7f790ffff640 1 -- 172.21.15.64:0/4136936560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7900003520 con 0x7f7900003f20 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.579+0000 7f790ffff640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900003f20 0x7f79000047c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79100ccf90 tx=0x7f79000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295117437058f1b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.579+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc005ad0 con 0x7f7900003f20 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.579+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc005c30 con 0x7f7900003f20 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.579+0000 7f790f7fe640 1 -- 172.21.15.64:0/4136936560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02c850 con 0x7f7900003f20 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.580+0000 7f7915a59640 1 -- 172.21.15.64:0/4136936560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900003f20 msgr2=0x7f79000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.580+0000 7f7915a59640 1 --2- 172.21.15.64:0/4136936560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900003f20 0x7f79000047c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79100ccf90 tx=0x7f79000057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.580+0000 7f7915258640 1 -- 172.21.15.64:0/4136936560 reap_dead start 2023-11-24T06:27:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.580+0000 7f7915a59640 1 -- 172.21.15.64:0/4136936560 shutdown_connections 2023-11-24T06:27:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.580+0000 7f7915a59640 1 -- 172.21.15.64:0/4136936560 >> 172.21.15.64:0/4136936560 conn(0x7f791006a7d0 msgr2=0x7f791006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.580+0000 7f7915a59640 1 -- 172.21.15.64:0/4136936560 shutdown_connections 2023-11-24T06:27:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.580+0000 7f7915a59640 1 -- 172.21.15.64:0/4136936560 wait complete. 2023-11-24T06:27:41.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.582+0000 7f7915a59640 1 Processor -- start 2023-11-24T06:27:41.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.582+0000 7f7915a59640 1 -- start start 2023-11-24T06:27:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915a59640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900003f20 0x7f79100af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915a59640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79100b1180 0x7f791013b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915a59640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79100b15e0 0x7f791013b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915a59640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7910140d70 con 0x7f79100b15e0 2023-11-24T06:27:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915a59640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7910140fc0 con 0x7f7900003f20 2023-11-24T06:27:41.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915a59640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79101412a0 con 0x7f79100b1180 2023-11-24T06:27:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915258640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79100b15e0 0x7f791013b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915258640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79100b15e0 0x7f791013b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53044/0 (socket says 172.21.15.64:53044) 2023-11-24T06:27:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915258640 1 -- 172.21.15.64:0/16221808 learned_addr learned my addr 172.21.15.64:0/16221808 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7914a57640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900003f20 0x7f79100af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f790ffff640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79100b1180 0x7f791013b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915258640 1 -- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79100b1180 msgr2=0x7f791013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.583+0000 7f7915258640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79100b1180 0x7f791013b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f7915258640 1 -- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900003f20 msgr2=0x7f79100af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f7915258640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900003f20 0x7f79100af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f7915258640 1 -- 172.21.15.64:0/16221808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910141580 con 0x7f79100b15e0 2023-11-24T06:27:41.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f790ffff640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79100b1180 0x7f791013b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:41.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f7914a57640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900003f20 0x7f79100af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:41.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f7915258640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79100b15e0 0x7f791013b6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f791015fb10 tx=0x7f79080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f790d7fa640 1 -- 172.21.15.64:0/16221808 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800e070 con 0x7f79100b15e0 2023-11-24T06:27:41.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f790d7fa640 1 -- 172.21.15.64:0/16221808 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908011e10 con 0x7f79100b15e0 2023-11-24T06:27:41.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f790d7fa640 1 -- 172.21.15.64:0/16221808 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908003990 con 0x7f79100b15e0 2023-11-24T06:27:41.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7910141860 con 0x7f79100b15e0 2023-11-24T06:27:41.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.584+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791015a990 con 0x7f79100b15e0 2023-11-24T06:27:41.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.585+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7910141d80 con 0x7f79100b15e0 2023-11-24T06:27:41.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.587+0000 7f790d7fa640 1 -- 172.21.15.64:0/16221808 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7908003af0 con 0x7f79100b15e0 2023-11-24T06:27:41.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.588+0000 7f790d7fa640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f78f406ae40 0x7f78f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.588+0000 7f790d7fa640 1 -- 172.21.15.64:0/16221808 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7908089ce0 con 0x7f79100b15e0 2023-11-24T06:27:41.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.588+0000 7f7914a57640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f78f406ae40 0x7f78f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.589+0000 7f7914a57640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f78f406ae40 0x7f78f406d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79040016b0 tx=0x7f7904009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.593+0000 7f790d7fa640 1 -- 172.21.15.64:0/16221808 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7910141d80 con 0x7f79100b15e0 2023-11-24T06:27:41.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:41.826+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7910141d80 con 0x7f79100b15e0 2023-11-24T06:27:42.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.754+0000 7f790d7fa640 1 -- 172.21.15.64:0/16221808 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7910141d80 con 0x7f79100b15e0 2023-11-24T06:27:42.757 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T06:27:42.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f78f406ae40 msgr2=0x7f78f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f78f406ae40 0x7f78f406d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79040016b0 tx=0x7f7904009040 comp rx=0 tx=0).stop 2023-11-24T06:27:42.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79100b15e0 msgr2=0x7f791013b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79100b15e0 0x7f791013b6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f791015fb10 tx=0x7f79080115a0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 shutdown_connections 2023-11-24T06:27:42.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f78f406ae40 0x7f78f406d300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79100b15e0 0x7f791013b6d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79100b1180 0x7f791013b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 --2- 172.21.15.64:0/16221808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7900003f20 0x7f79100af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 >> 172.21.15.64:0/16221808 conn(0x7f791006a7d0 msgr2=0x7f79100713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:42.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 shutdown_connections 2023-11-24T06:27:42.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.758+0000 7f7915a59640 1 -- 172.21.15.64:0/16221808 wait complete. 2023-11-24T06:27:42.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:42.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.996+0000 7f0f33fff640 1 Processor -- start 2023-11-24T06:27:42.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.996+0000 7f0f33fff640 1 -- start start 2023-11-24T06:27:43.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.998+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f34162430 0x7f0f340b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.998+0000 7f0f33fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f34162810 0x7f0f34071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.998+0000 7f0f33fff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f340702e0 con 0x7f0f340b3650 2023-11-24T06:27:43.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.998+0000 7f0f33fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f340705c0 con 0x7f0f34162810 2023-11-24T06:27:43.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.998+0000 7f0f33fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340708a0 con 0x7f0f34162430 2023-11-24T06:27:43.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.998+0000 7f0f327fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f34162810 0x7f0f34071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.999+0000 7f0f327fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f34162810 0x7f0f34071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48062/0 (socket says 172.21.15.64:48062) 2023-11-24T06:27:43.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.999+0000 7f0f327fc640 1 -- 172.21.15.64:0/1012207902 learned_addr learned my addr 172.21.15.64:0/1012207902 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:43.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.999+0000 7f0f32ffd640 1 --2- 172.21.15.64:0/1012207902 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f34162430 0x7f0f340b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.999+0000 7f0f327fc640 1 -- 172.21.15.64:0/1012207902 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f34162430 msgr2=0x7f0f340b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.999+0000 7f0f327fc640 1 --2- 172.21.15.64:0/1012207902 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f34162430 0x7f0f340b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.999+0000 7f0f327fc640 1 -- 172.21.15.64:0/1012207902 >> v1:172.21.15.162:6789/0 conn(0x7f0f340b3650 legacy=0x7f0f341650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:42.999+0000 7f0f327fc640 1 -- 172.21.15.64:0/1012207902 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f34070b80 con 0x7f0f34162810 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.000+0000 7f0f327fc640 1 --2- 172.21.15.64:0/1012207902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f34162810 0x7f0f34071240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009870 tx=0x7f0f2802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ffe3f281a0bbb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.000+0000 7f0f31ffb640 1 -- 172.21.15.64:0/1012207902 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28030840 con 0x7f0f34162810 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.000+0000 7f0f31ffb640 1 -- 172.21.15.64:0/1012207902 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f280309a0 con 0x7f0f34162810 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f31ffb640 1 -- 172.21.15.64:0/1012207902 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28030d30 con 0x7f0f34162810 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 -- 172.21.15.64:0/1012207902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f34162810 msgr2=0x7f0f34071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1012207902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f34162810 0x7f0f34071240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009870 tx=0x7f0f2802b980 comp rx=0 tx=0).stop 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 -- 172.21.15.64:0/1012207902 shutdown_connections 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1012207902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f34162810 0x7f0f34071240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1012207902 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f34162430 0x7f0f340b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 -- 172.21.15.64:0/1012207902 >> 172.21.15.64:0/1012207902 conn(0x7f0f3406a800 msgr2=0x7f0f3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 -- 172.21.15.64:0/1012207902 shutdown_connections 2023-11-24T06:27:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.001+0000 7f0f33fff640 1 -- 172.21.15.64:0/1012207902 wait complete. 2023-11-24T06:27:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.003+0000 7f0f33fff640 1 Processor -- start 2023-11-24T06:27:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.003+0000 7f0f33fff640 1 -- start start 2023-11-24T06:27:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.004+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f340b3650 0x7f0f3415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.004+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f34162430 0x7f0f341639f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.004+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f341609d0 0x7f0f34151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.004+0000 7f0f33fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34159840 con 0x7f0f340b3650 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.004+0000 7f0f33fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415bf00 con 0x7f0f34162430 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.004+0000 7f0f33fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415bcf0 con 0x7f0f341609d0 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.004+0000 7f0f32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f340b3650 0x7f0f3415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f340b3650 0x7f0f3415b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48064/0 (socket says 172.21.15.64:48064) 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 -- 172.21.15.64:0/1256253859 learned_addr learned my addr 172.21.15.64:0/1256253859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f327fc640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f34162430 0x7f0f341639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f337fe640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f341609d0 0x7f0f34151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 -- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f341609d0 msgr2=0x7f0f34151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f341609d0 0x7f0f34151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 -- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f34162430 msgr2=0x7f0f341639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f34162430 0x7f0f341639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 -- 172.21.15.64:0/1256253859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280094d0 con 0x7f0f340b3650 2023-11-24T06:27:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.005+0000 7f0f32ffd640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f340b3650 0x7f0f3415b7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d800 tx=0x7f0f2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.006+0000 7f0f1b7fe640 1 -- 172.21.15.64:0/1256253859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400b6c0 con 0x7f0f340b3650 2023-11-24T06:27:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.006+0000 7f0f1b7fe640 1 -- 172.21.15.64:0/1256253859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2400b840 con 0x7f0f340b3650 2023-11-24T06:27:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.006+0000 7f0f1b7fe640 1 -- 172.21.15.64:0/1256253859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400bbf0 con 0x7f0f340b3650 2023-11-24T06:27:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.006+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f341eca70 con 0x7f0f340b3650 2023-11-24T06:27:43.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.006+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f341ec420 con 0x7f0f340b3650 2023-11-24T06:27:43.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.007+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f340b2bf0 con 0x7f0f340b3650 2023-11-24T06:27:43.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.008+0000 7f0f1b7fe640 1 -- 172.21.15.64:0/1256253859 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f2400bd50 con 0x7f0f340b3650 2023-11-24T06:27:43.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.009+0000 7f0f1b7fe640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f2006ae70 0x7f0f2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.009+0000 7f0f1b7fe640 1 -- 172.21.15.64:0/1256253859 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f24089b00 con 0x7f0f340b3650 2023-11-24T06:27:43.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.009+0000 7f0f327fc640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f2006ae70 0x7f0f2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.010+0000 7f0f327fc640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f2006ae70 0x7f0f2006d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f28036000 tx=0x7f0f2803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.013+0000 7f0f1b7fe640 1 -- 172.21.15.64:0/1256253859 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f24054c40 con 0x7f0f340b3650 2023-11-24T06:27:43.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.217+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f34067450 con 0x7f0f340b3650 2023-11-24T06:27:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.220+0000 7f0f1b7fe640 1 -- 172.21.15.64:0/1256253859 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f0f24058260 con 0x7f0f340b3650 2023-11-24T06:27:43.223 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:43.223 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:41.827763+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:43.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f2006ae70 msgr2=0x7f0f2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f2006ae70 0x7f0f2006d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f28036000 tx=0x7f0f2803b040 comp rx=0 tx=0).stop 2023-11-24T06:27:43.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f340b3650 msgr2=0x7f0f3415b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f340b3650 0x7f0f3415b7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d800 tx=0x7f0f2400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 shutdown_connections 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f2006ae70 0x7f0f2006d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f341609d0 0x7f0f34151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f34162430 0x7f0f341639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 --2- 172.21.15.64:0/1256253859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f340b3650 0x7f0f3415b7b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 >> 172.21.15.64:0/1256253859 conn(0x7f0f3406a800 msgr2=0x7f0f340b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 shutdown_connections 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.223+0000 7f0f33fff640 1 -- 172.21.15.64:0/1256253859 wait complete. 2023-11-24T06:27:43.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.464+0000 7f6e1b59e640 1 Processor -- start 2023-11-24T06:27:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.464+0000 7f6e1b59e640 1 -- start start 2023-11-24T06:27:43.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.466+0000 7f6e1b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 0x7f6e1c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.466+0000 7f6e1b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1c0b2e10 con 0x7f6e1c071410 2023-11-24T06:27:43.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.466+0000 7f6e1b59e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1c06cec0 con 0x7f6e1c06c7c0 2023-11-24T06:27:43.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.466+0000 7f6e1b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c06d1a0 con 0x7f6e1c071f00 2023-11-24T06:27:43.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.466+0000 7f6e1a59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6e1c071410 0x7f6e1c0717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44688/0 (socket says 172.21.15.64:44688) 2023-11-24T06:27:43.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.466+0000 7f6e1a59c640 1 -- 172.21.15.64:0/2315050342 learned_addr learned my addr 172.21.15.64:0/2315050342 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469278660 0 0) 0x7f6e1c0b2e10 con 0x7f6e1c071410 2023-11-24T06:27:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e08003520 con 0x7f6e1c071410 2023-11-24T06:27:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e11d9b640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 0x7f6e1c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2452025598 0 0) 0x7f6e08003520 con 0x7f6e1c071410 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 msgr2=0x7f6e1c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 0x7f6e1c146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 >> v1:172.21.15.162:6789/0 conn(0x7f6e1c06c7c0 legacy=0x7f6e1c166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c06d480 con 0x7f6e1c071410 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f6e0c001670 con 0x7f6e1c071410 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 >> v1:172.21.15.64:6789/0 conn(0x7f6e1c071410 legacy=0x7f6e1c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e080039a0 0x7f6e08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.467+0000 7f6e19d9b640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 0x7f6e080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e19d9b640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e08004d00 0x7f6e08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c06d1a0 con 0x7f6e08004d00 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0800ba60 con 0x7f6e080039a0 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0800bc20 con 0x7f6e1c071f00 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e1ad9d640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e08004d00 0x7f6e08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e1a59c640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e080039a0 0x7f6e08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e11d9b640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 0x7f6e080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 msgr2=0x7f6e080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e1ad9d640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c071f00 0x7f6e080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e080039a0 msgr2=0x7f6e08003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e1ad9d640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e080039a0 0x7f6e08003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.468+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2315050342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08003520 con 0x7f6e08004d00 2023-11-24T06:27:43.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.469+0000 7f6e1ad9d640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e08004d00 0x7f6e08005180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e14001e20 tx=0x7f6e080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec3dc33be4fc86f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14003b70 con 0x7f6e08004d00 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e14003cd0 con 0x7f6e08004d00 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e19d9b640 1 -- 172.21.15.64:0/2315050342 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14005310 con 0x7f6e08004d00 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2315050342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e08004d00 msgr2=0x7f6e08005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e1b59e640 1 --2- 172.21.15.64:0/2315050342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e08004d00 0x7f6e08005180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e14001e20 tx=0x7f6e080057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2315050342 reap_dead start 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2315050342 shutdown_connections 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2315050342 >> 172.21.15.64:0/2315050342 conn(0x7f6e1c06a800 msgr2=0x7f6e1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2315050342 shutdown_connections 2023-11-24T06:27:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.470+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2315050342 wait complete. 2023-11-24T06:27:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.472+0000 7f6e1b59e640 1 Processor -- start 2023-11-24T06:27:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.472+0000 7f6e1b59e640 1 -- start start 2023-11-24T06:27:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.473+0000 7f6e1b59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e08004d00 0x7f6e1c13be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.473+0000 7f6e1b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c15f330 0x7f6e1c13b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.473+0000 7f6e1b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e1c141e80 0x7f6e1c142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.473+0000 7f6e1b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c142840 con 0x7f6e1c141e80 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.473+0000 7f6e1b59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c13cf80 con 0x7f6e08004d00 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.473+0000 7f6e1b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c13c6e0 con 0x7f6e1c15f330 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e1c141e80 0x7f6e1c142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e1c141e80 0x7f6e1c142300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48082/0 (socket says 172.21.15.64:48082) 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2787038417 learned_addr learned my addr 172.21.15.64:0/2787038417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e11d9b640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c15f330 0x7f6e1c13b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1a59c640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e08004d00 0x7f6e1c13be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c15f330 msgr2=0x7f6e1c13b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c15f330 0x7f6e1c13b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e08004d00 msgr2=0x7f6e1c13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e08004d00 0x7f6e1c13be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 -- 172.21.15.64:0/2787038417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c13c9c0 con 0x7f6e1c141e80 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e11d9b640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c15f330 0x7f6e1c13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1a59c640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e08004d00 0x7f6e1c13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.474+0000 7f6e1ad9d640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e1c141e80 0x7f6e1c142300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e14001f40 tx=0x7f6e140014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.475+0000 7f6e137fe640 1 -- 172.21.15.64:0/2787038417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1403e070 con 0x7f6e1c141e80 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.475+0000 7f6e137fe640 1 -- 172.21.15.64:0/2787038417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e14003e40 con 0x7f6e1c141e80 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.475+0000 7f6e137fe640 1 -- 172.21.15.64:0/2787038417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14006d90 con 0x7f6e1c141e80 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.475+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c13cca0 con 0x7f6e1c141e80 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.475+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c13c380 con 0x7f6e1c141e80 2023-11-24T06:27:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.476+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c0cbf40 con 0x7f6e1c141e80 2023-11-24T06:27:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.477+0000 7f6e137fe640 1 -- 172.21.15.64:0/2787038417 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e14007030 con 0x7f6e1c141e80 2023-11-24T06:27:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.478+0000 7f6e137fe640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6dfc06ae40 0x7f6dfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.478+0000 7f6e137fe640 1 -- 172.21.15.64:0/2787038417 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e140aa7a0 con 0x7f6e1c141e80 2023-11-24T06:27:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.478+0000 7f6e1a59c640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6dfc06ae40 0x7f6dfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.479+0000 7f6e1a59c640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6dfc06ae40 0x7f6dfc06d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c001eb0 tx=0x7f6e0c001d80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.484+0000 7f6e137fe640 1 -- 172.21.15.64:0/2787038417 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e1c0cbf40 con 0x7f6e1c141e80 2023-11-24T06:27:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.682+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e1c142840 con 0x7f6e1c141e80 2023-11-24T06:27:43.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.685+0000 7f6e137fe640 1 -- 172.21.15.64:0/2787038417 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f6e14048090 con 0x7f6e1c141e80 2023-11-24T06:27:43.690 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:43.690 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6dfc06ae40 msgr2=0x7f6dfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6dfc06ae40 0x7f6dfc06d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c001eb0 tx=0x7f6e0c001d80 comp rx=0 tx=0).stop 2023-11-24T06:27:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e1c141e80 msgr2=0x7f6e1c142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e1c141e80 0x7f6e1c142300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e14001f40 tx=0x7f6e140014a0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 shutdown_connections 2023-11-24T06:27:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6dfc06ae40 0x7f6dfc06d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e1c141e80 0x7f6e1c142300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e1c15f330 0x7f6e1c13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 --2- 172.21.15.64:0/2787038417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e08004d00 0x7f6e1c13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 >> 172.21.15.64:0/2787038417 conn(0x7f6e1c06a800 msgr2=0x7f6e1c145230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 shutdown_connections 2023-11-24T06:27:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.688+0000 7f6e1b59e640 1 -- 172.21.15.64:0/2787038417 wait complete. 2023-11-24T06:27:43.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:27:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.929+0000 7f05bdf4e640 1 Processor -- start 2023-11-24T06:27:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.929+0000 7f05bdf4e640 1 -- start start 2023-11-24T06:27:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.930+0000 7f05bdf4e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05b80adf20 0x7f05b8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.930+0000 7f05bdf4e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b815f4e0 con 0x7f05b815bf10 2023-11-24T06:27:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.930+0000 7f05bdf4e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b8071130 con 0x7f05b80ccb50 2023-11-24T06:27:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.930+0000 7f05bdf4e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b815f970 con 0x7f05b80adf20 2023-11-24T06:27:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.930+0000 7f05bcf4c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05b80adf20 0x7f05b8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.930+0000 7f05bcf4c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05b80adf20 0x7f05b8070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48096/0 (socket says 172.21.15.64:48096) 2023-11-24T06:27:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.930+0000 7f05bcf4c640 1 -- 172.21.15.64:0/2628697047 learned_addr learned my addr 172.21.15.64:0/2628697047 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.931+0000 7f05bcf4c640 1 -- 172.21.15.64:0/2628697047 >> v1:172.21.15.162:6789/0 conn(0x7f05b80ccb50 legacy=0x7f05b8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.931+0000 7f05bcf4c640 1 -- 172.21.15.64:0/2628697047 >> v1:172.21.15.64:6789/0 conn(0x7f05b815bf10 legacy=0x7f05b80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.931+0000 7f05bcf4c640 1 -- 172.21.15.64:0/2628697047 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b815c390 con 0x7f05b80adf20 2023-11-24T06:27:43.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.931+0000 7f05b77fe640 1 -- 172.21.15.64:0/2628697047 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329666947 0 0) 0x7f05b815f4e0 con 0x7f05b815bf10 2023-11-24T06:27:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.932+0000 7f05bcf4c640 1 --2- 172.21.15.64:0/2628697047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80adf20 0x7f05b8070a30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009e50 tx=0x7f05a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a30afc4fc3de87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.932+0000 7f05b77fe640 1 -- 172.21.15.64:0/2628697047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a802ca80 con 0x7f05b80adf20 2023-11-24T06:27:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.932+0000 7f05b77fe640 1 -- 172.21.15.64:0/2628697047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a802cbe0 con 0x7f05b80adf20 2023-11-24T06:27:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.932+0000 7f05b77fe640 1 -- 172.21.15.64:0/2628697047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80343f0 con 0x7f05b80adf20 2023-11-24T06:27:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.933+0000 7f05bdf4e640 1 -- 172.21.15.64:0/2628697047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80adf20 msgr2=0x7f05b8070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.933+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/2628697047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80adf20 0x7f05b8070a30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009e50 tx=0x7f05a802bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.933+0000 7f05bdf4e640 1 -- 172.21.15.64:0/2628697047 shutdown_connections 2023-11-24T06:27:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.933+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/2628697047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80adf20 0x7f05b8070a30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.933+0000 7f05bdf4e640 1 -- 172.21.15.64:0/2628697047 >> 172.21.15.64:0/2628697047 conn(0x7f05b806a7d0 msgr2=0x7f05b806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.933+0000 7f05bdf4e640 1 -- 172.21.15.64:0/2628697047 shutdown_connections 2023-11-24T06:27:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.933+0000 7f05bdf4e640 1 -- 172.21.15.64:0/2628697047 wait complete. 2023-11-24T06:27:43.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.935+0000 7f05bdf4e640 1 Processor -- start 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.935+0000 7f05bdf4e640 1 -- start start 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05bdf4e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05b80adf20 0x7f05b81eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05bdf4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80ccb50 0x7f05b80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05bdf4e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05b815bf10 0x7f05b80ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05bdf4e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81f2b00 con 0x7f05b80ccb50 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05bdf4e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81f6ed0 con 0x7f05b80adf20 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05bdf4e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81f2dc0 con 0x7f05b815bf10 2023-11-24T06:27:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80ccb50 0x7f05b80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80ccb50 0x7f05b80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48106/0 (socket says 172.21.15.64:48106) 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.936+0000 7f05b7fff640 1 -- 172.21.15.64:0/3522639328 learned_addr learned my addr 172.21.15.64:0/3522639328 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05bd74d640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05b815bf10 0x7f05b80ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05bcf4c640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05b80adf20 0x7f05b81eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05b7fff640 1 -- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05b815bf10 msgr2=0x7f05b80ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05b7fff640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05b815bf10 0x7f05b80ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05b7fff640 1 -- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05b80adf20 msgr2=0x7f05b81eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05b7fff640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05b80adf20 0x7f05b81eff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05b7fff640 1 -- 172.21.15.64:0/3522639328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009b30 con 0x7f05b80ccb50 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.937+0000 7f05b7fff640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80ccb50 0x7f05b80ac8d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c410 tx=0x7f05a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.938+0000 7f05b57fa640 1 -- 172.21.15.64:0/3522639328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a00035f0 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.938+0000 7f05b57fa640 1 -- 172.21.15.64:0/3522639328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a0003770 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.938+0000 7f05b57fa640 1 -- 172.21.15.64:0/3522639328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0003b20 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.938+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b81f30a0 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.938+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b81f04c0 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.939+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b81f3560 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.940+0000 7f05b57fa640 1 -- 172.21.15.64:0/3522639328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05a0003c80 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.941+0000 7f05b57fa640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f059806ae40 0x7f059806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.941+0000 7f05b57fa640 1 -- 172.21.15.64:0/3522639328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05a0088c40 con 0x7f05b80ccb50 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.941+0000 7f05bcf4c640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f059806ae40 0x7f059806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.942+0000 7f05bcf4c640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f059806ae40 0x7f059806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f05a8035000 tx=0x7f05a8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:43.945+0000 7f05b57fa640 1 -- 172.21.15.64:0/3522639328 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05a0053ce0 con 0x7f05b80ccb50 2023-11-24T06:27:44.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.178+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f05b8067450 con 0x7f05b80ccb50 2023-11-24T06:27:44.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.766+0000 7f05b57fa640 1 -- 172.21.15.64:0/3522639328 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f05a0017020 con 0x7f05b80ccb50 2023-11-24T06:27:44.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f059806ae40 msgr2=0x7f059806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f059806ae40 0x7f059806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f05a8035000 tx=0x7f05a8030000 comp rx=0 tx=0).stop 2023-11-24T06:27:44.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80ccb50 msgr2=0x7f05b80ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80ccb50 0x7f05b80ac8d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c410 tx=0x7f05a000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 shutdown_connections 2023-11-24T06:27:44.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f059806ae40 0x7f059806d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05b815bf10 0x7f05b80ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05b80ccb50 0x7f05b80ac8d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 --2- 172.21.15.64:0/3522639328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05b80adf20 0x7f05b81eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 >> 172.21.15.64:0/3522639328 conn(0x7f05b806a7d0 msgr2=0x7f05b815b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:44.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 shutdown_connections 2023-11-24T06:27:44.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:44.771+0000 7f05bdf4e640 1 -- 172.21.15.64:0/3522639328 wait complete. 2023-11-24T06:27:44.796 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:27:44.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:27:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.024+0000 7f383759e640 1 Processor -- start 2023-11-24T06:27:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.024+0000 7f383759e640 1 -- start start 2023-11-24T06:27:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.026+0000 7f383759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f38380b35a0 0x7f38380cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.026+0000 7f383759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38380ce440 con 0x7f383813cf10 2023-11-24T06:27:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.026+0000 7f383759e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38380ce720 con 0x7f383813fcb0 2023-11-24T06:27:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.026+0000 7f383759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38380cea00 con 0x7f38380b35a0 2023-11-24T06:27:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f3836d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f38380b35a0 0x7f38380cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f3836d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f38380b35a0 0x7f38380cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48108/0 (socket says 172.21.15.64:48108) 2023-11-24T06:27:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f3836d9d640 1 -- 172.21.15.64:0/623317283 learned_addr learned my addr 172.21.15.64:0/623317283 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f383659c640 1 --1- 172.21.15.64:0/623317283 >> v1:172.21.15.64:6789/0 conn(0x7f383813cf10 0x7f383813f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44700/0 (socket says 172.21.15.64:44700) 2023-11-24T06:27:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f3836d9d640 1 -- 172.21.15.64:0/623317283 >> v1:172.21.15.162:6789/0 conn(0x7f383813fcb0 legacy=0x7f38380cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f3836d9d640 1 -- 172.21.15.64:0/623317283 >> v1:172.21.15.64:6789/0 conn(0x7f383813cf10 legacy=0x7f383813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f3836d9d640 1 -- 172.21.15.64:0/623317283 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380b1d90 con 0x7f38380b35a0 2023-11-24T06:27:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.027+0000 7f383559a640 1 -- 172.21.15.64:0/623317283 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298788610 0 0) 0x7f38380ce440 con 0x7f383813cf10 2023-11-24T06:27:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.028+0000 7f3836d9d640 1 --2- 172.21.15.64:0/623317283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38380b35a0 0x7f38380cddf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c180ec72e8464792 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383559a640 1 -- 172.21.15.64:0/623317283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383002cbb0 con 0x7f38380b35a0 2023-11-24T06:27:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383559a640 1 -- 172.21.15.64:0/623317283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383002cd10 con 0x7f38380b35a0 2023-11-24T06:27:45.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383559a640 1 -- 172.21.15.64:0/623317283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830034610 con 0x7f38380b35a0 2023-11-24T06:27:45.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383759e640 1 -- 172.21.15.64:0/623317283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38380b35a0 msgr2=0x7f38380cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383759e640 1 --2- 172.21.15.64:0/623317283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38380b35a0 0x7f38380cddf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383002c010 comp rx=0 tx=0).stop 2023-11-24T06:27:45.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383759e640 1 -- 172.21.15.64:0/623317283 shutdown_connections 2023-11-24T06:27:45.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383759e640 1 --2- 172.21.15.64:0/623317283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38380b35a0 0x7f38380cddf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.029+0000 7f383759e640 1 -- 172.21.15.64:0/623317283 >> 172.21.15.64:0/623317283 conn(0x7f383806a7d0 msgr2=0x7f383806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.030+0000 7f383759e640 1 -- 172.21.15.64:0/623317283 shutdown_connections 2023-11-24T06:27:45.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.030+0000 7f383759e640 1 -- 172.21.15.64:0/623317283 wait complete. 2023-11-24T06:27:45.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.031+0000 7f383759e640 1 Processor -- start 2023-11-24T06:27:45.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.031+0000 7f383759e640 1 -- start start 2023-11-24T06:27:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.032+0000 7f383759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38380b35a0 0x7f38381388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.032+0000 7f383759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383813cf10 0x7f38381522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.032+0000 7f383759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f383813fcb0 0x7f383813e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.032+0000 7f383759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838151320 con 0x7f383813cf10 2023-11-24T06:27:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.032+0000 7f383759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383813e1e0 con 0x7f38380b35a0 2023-11-24T06:27:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.032+0000 7f383759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383813d8b0 con 0x7f383813fcb0 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.032+0000 7f3835d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383813cf10 0x7f38381522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383813cf10 0x7f38381522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48120/0 (socket says 172.21.15.64:48120) 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 -- 172.21.15.64:0/2542903560 learned_addr learned my addr 172.21.15.64:0/2542903560 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f383659c640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38380b35a0 0x7f38381388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3836d9d640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f383813fcb0 0x7f383813e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 -- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f383813fcb0 msgr2=0x7f383813e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f383813fcb0 0x7f383813e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 -- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38380b35a0 msgr2=0x7f38381388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38380b35a0 0x7f38381388e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 -- 172.21.15.64:0/2542903560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830009b30 con 0x7f383813cf10 2023-11-24T06:27:45.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3836d9d640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f383813fcb0 0x7f383813e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f383659c640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38380b35a0 0x7f38381388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.033+0000 7f3835d9b640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383813cf10 0x7f38381522b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c400 tx=0x7f382c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.034+0000 7f3826ffd640 1 -- 172.21.15.64:0/2542903560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0052d0 con 0x7f383813cf10 2023-11-24T06:27:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.034+0000 7f3826ffd640 1 -- 172.21.15.64:0/2542903560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c005430 con 0x7f383813cf10 2023-11-24T06:27:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.034+0000 7f3826ffd640 1 -- 172.21.15.64:0/2542903560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c003830 con 0x7f383813cf10 2023-11-24T06:27:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.034+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383813db90 con 0x7f383813cf10 2023-11-24T06:27:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.034+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383813d4d0 con 0x7f383813cf10 2023-11-24T06:27:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.035+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383813e390 con 0x7f383813cf10 2023-11-24T06:27:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.036+0000 7f3826ffd640 1 -- 172.21.15.64:0/2542903560 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f382c003990 con 0x7f383813cf10 2023-11-24T06:27:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.037+0000 7f3826ffd640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f381806ae40 0x7f381806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.037+0000 7f3826ffd640 1 -- 172.21.15.64:0/2542903560 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f382c088ae0 con 0x7f383813cf10 2023-11-24T06:27:45.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.037+0000 7f383659c640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f381806ae40 0x7f381806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.038+0000 7f383659c640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f381806ae40 0x7f381806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3828000f60 tx=0x7f3828009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.042+0000 7f3826ffd640 1 -- 172.21.15.64:0/2542903560 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f383813e390 con 0x7f383813cf10 2023-11-24T06:27:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.281+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f383813e350 con 0x7f383813cf10 2023-11-24T06:27:45.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.769+0000 7f3826ffd640 1 -- 172.21.15.64:0/2542903560 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f382c0170c0 con 0x7f383813cf10 2023-11-24T06:27:45.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f381806ae40 msgr2=0x7f381806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f381806ae40 0x7f381806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3828000f60 tx=0x7f3828009040 comp rx=0 tx=0).stop 2023-11-24T06:27:45.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383813cf10 msgr2=0x7f38381522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383813cf10 0x7f38381522b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c400 tx=0x7f382c00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 shutdown_connections 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f381806ae40 0x7f381806d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f383813fcb0 0x7f383813e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383813cf10 0x7f38381522b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 --2- 172.21.15.64:0/2542903560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38380b35a0 0x7f38381388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 >> 172.21.15.64:0/2542903560 conn(0x7f383806a7d0 msgr2=0x7f383815b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 shutdown_connections 2023-11-24T06:27:45.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:45.775+0000 7f383759e640 1 -- 172.21.15.64:0/2542903560 wait complete. 2023-11-24T06:27:45.799 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:27:45.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.013+0000 7ff91f26d640 1 Processor -- start 2023-11-24T06:27:46.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.013+0000 7ff91f26d640 1 -- start start 2023-11-24T06:27:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.015+0000 7ff91f26d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff918144180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.015+0000 7ff91f26d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9180ce600 0x7ff918072210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.015+0000 7ff91f26d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9180ceab0 con 0x7ff918143da0 2023-11-24T06:27:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.015+0000 7ff91f26d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91806c7c0 con 0x7ff9180ce600 2023-11-24T06:27:46.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.015+0000 7ff91f26d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91815b4e0 con 0x7ff9180cca00 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91da6a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff918144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91ea6c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9180ce600 0x7ff918072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91da6a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff918144180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:60928/0 (socket says 172.21.15.64:60928) 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91da6a640 1 -- 172.21.15.64:0/1360202799 learned_addr learned my addr 172.21.15.64:0/1360202799 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91da6a640 1 -- 172.21.15.64:0/1360202799 >> v2:172.21.15.162:3300/0 conn(0x7ff9180ce600 msgr2=0x7ff918072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91da6a640 1 --2- 172.21.15.64:0/1360202799 >> v2:172.21.15.162:3300/0 conn(0x7ff9180ce600 0x7ff918072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91da6a640 1 -- 172.21.15.64:0/1360202799 >> v1:172.21.15.162:6789/0 conn(0x7ff918143da0 legacy=0x7ff9180cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.016+0000 7ff91da6a640 1 -- 172.21.15.64:0/1360202799 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918071690 con 0x7ff9180cca00 2023-11-24T06:27:46.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.018+0000 7ff91da6a640 1 --2- 172.21.15.64:0/1360202799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff918144180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff908009870 tx=0x7ff90802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3605cda995ebea35 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.018+0000 7ff91d269640 1 -- 172.21.15.64:0/1360202799 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908031640 con 0x7ff9180cca00 2023-11-24T06:27:46.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.018+0000 7ff91d269640 1 -- 172.21.15.64:0/1360202799 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9080317a0 con 0x7ff9180cca00 2023-11-24T06:27:46.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.018+0000 7ff91d269640 1 -- 172.21.15.64:0/1360202799 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908031b30 con 0x7ff9180cca00 2023-11-24T06:27:46.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 -- 172.21.15.64:0/1360202799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 msgr2=0x7ff918144180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 --2- 172.21.15.64:0/1360202799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff918144180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff908009870 tx=0x7ff90802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 -- 172.21.15.64:0/1360202799 shutdown_connections 2023-11-24T06:27:46.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 --2- 172.21.15.64:0/1360202799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff918144180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 --2- 172.21.15.64:0/1360202799 >> v2:172.21.15.162:3300/0 conn(0x7ff9180ce600 0x7ff918072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 -- 172.21.15.64:0/1360202799 >> 172.21.15.64:0/1360202799 conn(0x7ff91806a800 msgr2=0x7ff91806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 -- 172.21.15.64:0/1360202799 shutdown_connections 2023-11-24T06:27:46.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.019+0000 7ff91f26d640 1 -- 172.21.15.64:0/1360202799 wait complete. 2023-11-24T06:27:46.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.021+0000 7ff91f26d640 1 Processor -- start 2023-11-24T06:27:46.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.021+0000 7ff91f26d640 1 -- start start 2023-11-24T06:27:46.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.022+0000 7ff91f26d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff9181384a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.022+0000 7ff91f26d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9180ce600 0x7ff91813be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.022+0000 7ff91f26d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff918143da0 0x7ff91813e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.022+0000 7ff91f26d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91813eed0 con 0x7ff918143da0 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.022+0000 7ff91f26d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91813f1b0 con 0x7ff9180ce600 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.022+0000 7ff91f26d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91813f490 con 0x7ff9180cca00 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff918143da0 0x7ff91813e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff918143da0 0x7ff91813e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48132/0 (socket says 172.21.15.64:48132) 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 -- 172.21.15.64:0/3602984455 learned_addr learned my addr 172.21.15.64:0/3602984455 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91e26b640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff9181384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91da6a640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9180ce600 0x7ff91813be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 -- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 msgr2=0x7ff9181384a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff9181384a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 -- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9180ce600 msgr2=0x7ff91813be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9180ce600 0x7ff91813be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91ea6c640 1 -- 172.21.15.64:0/3602984455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080094d0 con 0x7ff918143da0 2023-11-24T06:27:46.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.023+0000 7ff91da6a640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9180ce600 0x7ff91813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:46.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.024+0000 7ff91ea6c640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff918143da0 0x7ff91813e090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff91000efc0 tx=0x7ff91000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.024+0000 7ff90effd640 1 -- 172.21.15.64:0/3602984455 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910014480 con 0x7ff918143da0 2023-11-24T06:27:46.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.024+0000 7ff90effd640 1 -- 172.21.15.64:0/3602984455 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91000f040 con 0x7ff918143da0 2023-11-24T06:27:46.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.024+0000 7ff90effd640 1 -- 172.21.15.64:0/3602984455 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910014820 con 0x7ff918143da0 2023-11-24T06:27:46.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.024+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91813f770 con 0x7ff918143da0 2023-11-24T06:27:46.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.024+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91815ab20 con 0x7ff918143da0 2023-11-24T06:27:46.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.025+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9181396a0 con 0x7ff918143da0 2023-11-24T06:27:46.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.026+0000 7ff90effd640 1 -- 172.21.15.64:0/3602984455 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff910014980 con 0x7ff918143da0 2023-11-24T06:27:46.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.027+0000 7ff90effd640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff8fc06ae40 0x7ff8fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.027+0000 7ff90effd640 1 -- 172.21.15.64:0/3602984455 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff91008a250 con 0x7ff918143da0 2023-11-24T06:27:46.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.027+0000 7ff91e26b640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff8fc06ae40 0x7ff8fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.028+0000 7ff91e26b640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff8fc06ae40 0x7ff8fc06d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff914000f60 tx=0x7ff914009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.031+0000 7ff90effd640 1 -- 172.21.15.64:0/3602984455 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9181396a0 con 0x7ff918143da0 2023-11-24T06:27:46.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.266+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9181396a0 con 0x7ff918143da0 2023-11-24T06:27:46.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.267+0000 7ff90effd640 1 -- 172.21.15.64:0/3602984455 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7ff9181396a0 con 0x7ff918143da0 2023-11-24T06:27:46.272 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:46.273 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:46.273 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff8fc06ae40 msgr2=0x7ff8fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff8fc06ae40 0x7ff8fc06d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff914000f60 tx=0x7ff914009040 comp rx=0 tx=0).stop 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff918143da0 msgr2=0x7ff91813e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff918143da0 0x7ff91813e090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff91000efc0 tx=0x7ff91000c490 comp rx=0 tx=0).stop 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 shutdown_connections 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff8fc06ae40 0x7ff8fc06d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff918143da0 0x7ff91813e090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9180ce600 0x7ff91813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 --2- 172.21.15.64:0/3602984455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9180cca00 0x7ff9181384a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 >> 172.21.15.64:0/3602984455 conn(0x7ff91806a800 msgr2=0x7ff9180cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 shutdown_connections 2023-11-24T06:27:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.270+0000 7ff91f26d640 1 -- 172.21.15.64:0/3602984455 wait complete. 2023-11-24T06:27:46.290 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:46.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.510+0000 7f831359e640 1 Processor -- start 2023-11-24T06:27:46.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.510+0000 7f831359e640 1 -- start start 2023-11-24T06:27:46.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.511+0000 7f831359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8314160920 0x7f83140b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.511+0000 7f831359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 0x7f83140cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.511+0000 7f831359e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83140d0a10 con 0x7f8314161010 2023-11-24T06:27:46.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83140d03b0 con 0x7f8314160920 2023-11-24T06:27:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83140d1f50 con 0x7f83140cf9f0 2023-11-24T06:27:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8314160920 0x7f83140b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8314160920 0x7f83140b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48136/0 (socket says 172.21.15.64:48136) 2023-11-24T06:27:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831259c640 1 -- 172.21.15.64:0/3071352848 learned_addr learned my addr 172.21.15.64:0/3071352848 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831259c640 1 -- 172.21.15.64:0/3071352848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 msgr2=0x7f83140cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831259c640 1 --2- 172.21.15.64:0/3071352848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 0x7f83140cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831259c640 1 -- 172.21.15.64:0/3071352848 >> v1:172.21.15.64:6789/0 conn(0x7f8314161010 legacy=0x7f83141614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.512+0000 7f831259c640 1 -- 172.21.15.64:0/3071352848 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140cf080 con 0x7f8314160920 2023-11-24T06:27:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.513+0000 7f831259c640 1 --2- 172.21.15.64:0/3071352848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314160920 0x7f83140b3f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8304009e50 tx=0x7f830402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692a86b8293ccd4e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831159a640 1 -- 172.21.15.64:0/3071352848 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830402cb00 con 0x7f8314160920 2023-11-24T06:27:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831159a640 1 -- 172.21.15.64:0/3071352848 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830402cc60 con 0x7f8314160920 2023-11-24T06:27:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831159a640 1 -- 172.21.15.64:0/3071352848 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83040343f0 con 0x7f8314160920 2023-11-24T06:27:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 -- 172.21.15.64:0/3071352848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314160920 msgr2=0x7f83140b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 --2- 172.21.15.64:0/3071352848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314160920 0x7f83140b3f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8304009e50 tx=0x7f830402bf60 comp rx=0 tx=0).stop 2023-11-24T06:27:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 -- 172.21.15.64:0/3071352848 shutdown_connections 2023-11-24T06:27:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 --2- 172.21.15.64:0/3071352848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 0x7f83140cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 --2- 172.21.15.64:0/3071352848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314160920 0x7f83140b3f30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 -- 172.21.15.64:0/3071352848 >> 172.21.15.64:0/3071352848 conn(0x7f83140d3090 msgr2=0x7f83140d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 -- 172.21.15.64:0/3071352848 shutdown_connections 2023-11-24T06:27:46.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.514+0000 7f831359e640 1 -- 172.21.15.64:0/3071352848 wait complete. 2023-11-24T06:27:46.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.516+0000 7f831359e640 1 Processor -- start 2023-11-24T06:27:46.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.516+0000 7f831359e640 1 -- start start 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.517+0000 7f831359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 0x7f8314154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.517+0000 7f831359e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8314160920 0x7f8314154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.517+0000 7f831359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314161010 0x7f83140b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.517+0000 7f831359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83140b71e0 con 0x7f8314161010 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.517+0000 7f831359e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83140cb600 con 0x7f8314160920 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.517+0000 7f831359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83140cb8e0 con 0x7f83140cf9f0 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314161010 0x7f83140b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314161010 0x7f83140b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48146/0 (socket says 172.21.15.64:48146) 2023-11-24T06:27:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 -- 172.21.15.64:0/3814923681 learned_addr learned my addr 172.21.15.64:0/3814923681 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f831259c640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 0x7f8314154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8311d9b640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8314160920 0x7f8314154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 -- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 msgr2=0x7f8314154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 0x7f8314154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 -- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8314160920 msgr2=0x7f8314154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8314160920 0x7f8314154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 -- 172.21.15.64:0/3814923681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304009b30 con 0x7f8314161010 2023-11-24T06:27:46.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.518+0000 7f8312d9d640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314161010 0x7f83140b6890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c390 tx=0x7f830c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.519+0000 7f830affd640 1 -- 172.21.15.64:0/3814923681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c0052d0 con 0x7f8314161010 2023-11-24T06:27:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.519+0000 7f830affd640 1 -- 172.21.15.64:0/3814923681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c005430 con 0x7f8314161010 2023-11-24T06:27:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.519+0000 7f830affd640 1 -- 172.21.15.64:0/3814923681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c003810 con 0x7f8314161010 2023-11-24T06:27:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.519+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83140cbbc0 con 0x7f8314161010 2023-11-24T06:27:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.519+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83140b74c0 con 0x7f8314161010 2023-11-24T06:27:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.520+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83140cbde0 con 0x7f8314161010 2023-11-24T06:27:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.521+0000 7f830affd640 1 -- 172.21.15.64:0/3814923681 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f830c003970 con 0x7f8314161010 2023-11-24T06:27:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.522+0000 7f830affd640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82f406ae40 0x7f82f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.522+0000 7f830affd640 1 -- 172.21.15.64:0/3814923681 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f830c088b40 con 0x7f8314161010 2023-11-24T06:27:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.522+0000 7f831259c640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82f406ae40 0x7f82f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.523+0000 7f831259c640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82f406ae40 0x7f82f406d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8304035000 tx=0x7f8304030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.527+0000 7f830affd640 1 -- 172.21.15.64:0/3814923681 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f830c053be0 con 0x7f8314161010 2023-11-24T06:27:46.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.725+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8314067450 con 0x7f8314161010 2023-11-24T06:27:46.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.727+0000 7f830affd640 1 -- 172.21.15.64:0/3814923681 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f830c017090 con 0x7f8314161010 2023-11-24T06:27:46.731 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:46.731 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82f406ae40 msgr2=0x7f82f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82f406ae40 0x7f82f406d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8304035000 tx=0x7f8304030000 comp rx=0 tx=0).stop 2023-11-24T06:27:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314161010 msgr2=0x7f83140b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314161010 0x7f83140b6890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c390 tx=0x7f830c00c850 comp rx=0 tx=0).stop 2023-11-24T06:27:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 shutdown_connections 2023-11-24T06:27:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82f406ae40 0x7f82f406d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314161010 0x7f83140b6890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8314160920 0x7f8314154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 --2- 172.21.15.64:0/3814923681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140cf9f0 0x7f8314154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 >> 172.21.15.64:0/3814923681 conn(0x7f83140d3090 msgr2=0x7f83140b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.730+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 shutdown_connections 2023-11-24T06:27:46.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.731+0000 7f831359e640 1 -- 172.21.15.64:0/3814923681 wait complete. 2023-11-24T06:27:46.754 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:27:46.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:46.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.973+0000 7f1b9ebe4640 1 Processor -- start 2023-11-24T06:27:46.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.973+0000 7f1b9ebe4640 1 -- start start 2023-11-24T06:27:46.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.974+0000 7f1b9ebe4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1b98143c40 0x7f1b98071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.974+0000 7f1b9ebe4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980717d0 0x7f1b980cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.974+0000 7f1b9ebe4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b98070e50 con 0x7f1b980cca40 2023-11-24T06:27:46.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.974+0000 7f1b9ebe4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980723c0 con 0x7f1b98143c40 2023-11-24T06:27:46.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.974+0000 7f1b9ebe4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980cce70 con 0x7f1b980717d0 2023-11-24T06:27:46.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.975+0000 7f1b9e3e3640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1b980cca40 0x7f1b9815eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44718/0 (socket says 172.21.15.64:44718) 2023-11-24T06:27:46.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.975+0000 7f1b9e3e3640 1 -- 172.21.15.64:0/854562370 learned_addr learned my addr 172.21.15.64:0/854562370 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:46.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.975+0000 7f1b9dbe2640 1 --2- 172.21.15.64:0/854562370 >> v2:172.21.15.162:3300/0 conn(0x7f1b98143c40 0x7f1b98071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.975+0000 7f1b9d3e1640 1 --2- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980717d0 0x7f1b980cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.975+0000 7f1b9cbe0640 1 -- 172.21.15.64:0/854562370 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165294738 0 0) 0x7f1b98070e50 con 0x7f1b980cca40 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.975+0000 7f1b9cbe0640 1 -- 172.21.15.64:0/854562370 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b84003520 con 0x7f1b980cca40 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.976+0000 7f1b9dbe2640 1 -- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980717d0 msgr2=0x7f1b980cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.976+0000 7f1b9dbe2640 1 --2- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980717d0 0x7f1b980cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.976+0000 7f1b9dbe2640 1 -- 172.21.15.64:0/854562370 >> v1:172.21.15.64:6789/0 conn(0x7f1b980cca40 legacy=0x7f1b9815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.976+0000 7f1b9dbe2640 1 -- 172.21.15.64:0/854562370 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9806e170 con 0x7f1b98143c40 2023-11-24T06:27:46.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.977+0000 7f1b9dbe2640 1 --2- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b98143c40 0x7f1b98071290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b940097a0 tx=0x7f1b9402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df9478779aacd21 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.978+0000 7f1b9cbe0640 1 -- 172.21.15.64:0/854562370 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94031780 con 0x7f1b98143c40 2023-11-24T06:27:46.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.978+0000 7f1b9cbe0640 1 -- 172.21.15.64:0/854562370 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b940318e0 con 0x7f1b98143c40 2023-11-24T06:27:46.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.978+0000 7f1b9cbe0640 1 -- 172.21.15.64:0/854562370 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94031c70 con 0x7f1b98143c40 2023-11-24T06:27:46.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.978+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b98143c40 msgr2=0x7f1b98071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.978+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b98143c40 0x7f1b98071290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b940097a0 tx=0x7f1b9402b820 comp rx=0 tx=0).stop 2023-11-24T06:27:46.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.979+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/854562370 shutdown_connections 2023-11-24T06:27:46.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.979+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980717d0 0x7f1b980cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.979+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/854562370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b98143c40 0x7f1b98071290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.979+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/854562370 >> 172.21.15.64:0/854562370 conn(0x7f1b9806a800 msgr2=0x7f1b9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.979+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/854562370 shutdown_connections 2023-11-24T06:27:46.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.979+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/854562370 wait complete. 2023-11-24T06:27:46.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.980+0000 7f1b9ebe4640 1 Processor -- start 2023-11-24T06:27:46.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.981+0000 7f1b9ebe4640 1 -- start start 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9ebe4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980717d0 0x7f1b9815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9ebe4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b980cca40 0x7f1b981e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9ebe4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b981e5740 0x7f1b981e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9ebe4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9815af50 con 0x7f1b980cca40 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9ebe4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981e5bd0 con 0x7f1b980717d0 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9ebe4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981e1bf0 con 0x7f1b981e5740 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b980cca40 0x7f1b981e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b980cca40 0x7f1b981e5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48160/0 (socket says 172.21.15.64:48160) 2023-11-24T06:27:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 -- 172.21.15.64:0/2928196992 learned_addr learned my addr 172.21.15.64:0/2928196992 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9e3e3640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b981e5740 0x7f1b981e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9dbe2640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980717d0 0x7f1b9815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 -- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b981e5740 msgr2=0x7f1b981e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b981e5740 0x7f1b981e0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 -- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980717d0 msgr2=0x7f1b9815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980717d0 0x7f1b9815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.982+0000 7f1b9d3e1640 1 -- 172.21.15.64:0/2928196992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b94009400 con 0x7f1b980cca40 2023-11-24T06:27:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.983+0000 7f1b9d3e1640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b980cca40 0x7f1b981e5200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e9b0 tx=0x7f1b8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.983+0000 7f1b8e7fc640 1 -- 172.21.15.64:0/2928196992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88009670 con 0x7f1b980cca40 2023-11-24T06:27:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.983+0000 7f1b8e7fc640 1 -- 172.21.15.64:0/2928196992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8800c9a0 con 0x7f1b980cca40 2023-11-24T06:27:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.983+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b981e1910 con 0x7f1b980cca40 2023-11-24T06:27:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.983+0000 7f1b8e7fc640 1 -- 172.21.15.64:0/2928196992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800cd50 con 0x7f1b980cca40 2023-11-24T06:27:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.983+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b981e2b30 con 0x7f1b980cca40 2023-11-24T06:27:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.984+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98067450 con 0x7f1b980cca40 2023-11-24T06:27:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.985+0000 7f1b8e7fc640 1 -- 172.21.15.64:0/2928196992 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b88013470 con 0x7f1b980cca40 2023-11-24T06:27:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.987+0000 7f1b8e7fc640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1b7c06ae40 0x7f1b7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.987+0000 7f1b8e7fc640 1 -- 172.21.15.64:0/2928196992 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b88089950 con 0x7f1b980cca40 2023-11-24T06:27:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.987+0000 7f1b9dbe2640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1b7c06ae40 0x7f1b7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.988+0000 7f1b9dbe2640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1b7c06ae40 0x7f1b7c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b94036000 tx=0x7f1b9403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:46.992+0000 7f1b8e7fc640 1 -- 172.21.15.64:0/2928196992 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b88054970 con 0x7f1b980cca40 2023-11-24T06:27:47.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.191+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b980720e0 con 0x7f1b7c06ae40 2023-11-24T06:27:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.209+0000 7f1b8e7fc640 1 -- 172.21.15.64:0/2928196992 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264500 (secure 0 0 0) 0x7f1b94031a50 con 0x7f1b7c06ae40 2023-11-24T06:27:47.220 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:47.221 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:27:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1b7c06ae40 msgr2=0x7f1b7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1b7c06ae40 0x7f1b7c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b94036000 tx=0x7f1b9403d040 comp rx=0 tx=0).stop 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b980cca40 msgr2=0x7f1b981e5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b980cca40 0x7f1b981e5200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e9b0 tx=0x7f1b8800ee70 comp rx=0 tx=0).stop 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 shutdown_connections 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1b7c06ae40 0x7f1b7c06d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b981e5740 0x7f1b981e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b980cca40 0x7f1b981e5200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 --2- 172.21.15.64:0/2928196992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980717d0 0x7f1b9815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 >> 172.21.15.64:0/2928196992 conn(0x7f1b9806a800 msgr2=0x7f1b980aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 shutdown_connections 2023-11-24T06:27:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.216+0000 7f1b9ebe4640 1 -- 172.21.15.64:0/2928196992 wait complete. 2023-11-24T06:27:47.240 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T06:27:47.102569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219312,"kb_used_data":2972,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755408,"statfs":{"total":773094113280,"available":772869537792,"internally_reserved":0,"allocated":3043328,"data_stored":1809672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002124"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658921+0000","last_change":"2023-11-24T06:27:42.394622+0000","last_active":"2023-11-24T06:27:43.658921+0000","last_peered":"2023-11-24T06:27:43.658921+0000","last_clean":"2023-11-24T06:27:43.658921+0000","last_became_active":"2023-11-24T06:27:42.394278+0000","last_became_peered":"2023-11-24T06:27:42.394278+0000","last_unstale":"2023-11-24T06:27:43.658921+0000","last_undegraded":"2023-11-24T06:27:43.658921+0000","last_fullsized":"2023-11-24T06:27:43.658921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659042+0000","last_change":"2023-11-24T06:27:42.395699+0000","last_active":"2023-11-24T06:27:43.659042+0000","last_peered":"2023-11-24T06:27:43.659042+0000","last_clean":"2023-11-24T06:27:43.659042+0000","last_became_active":"2023-11-24T06:27:42.395424+0000","last_became_peered":"2023-11-24T06:27:42.395424+0000","last_unstale":"2023-11-24T06:27:43.659042+0000","last_undegraded":"2023-11-24T06:27:43.659042+0000","last_fullsized":"2023-11-24T06:27:43.659042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660634+0000","last_change":"2023-11-24T06:27:42.390431+0000","last_active":"2023-11-24T06:27:43.660634+0000","last_peered":"2023-11-24T06:27:43.660634+0000","last_clean":"2023-11-24T06:27:43.660634+0000","last_became_active":"2023-11-24T06:27:42.390163+0000","last_became_peered":"2023-11-24T06:27:42.390163+0000","last_unstale":"2023-11-24T06:27:43.660634+0000","last_undegraded":"2023-11-24T06:27:43.660634+0000","last_fullsized":"2023-11-24T06:27:43.660634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357785+0000","last_change":"2023-11-24T06:27:42.393940+0000","last_active":"2023-11-24T06:27:43.357785+0000","last_peered":"2023-11-24T06:27:43.357785+0000","last_clean":"2023-11-24T06:27:43.357785+0000","last_became_active":"2023-11-24T06:27:42.393341+0000","last_became_peered":"2023-11-24T06:27:42.393341+0000","last_unstale":"2023-11-24T06:27:43.357785+0000","last_undegraded":"2023-11-24T06:27:43.357785+0000","last_fullsized":"2023-11-24T06:27:43.357785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658343+0000","last_change":"2023-11-24T06:27:42.392760+0000","last_active":"2023-11-24T06:27:43.658343+0000","last_peered":"2023-11-24T06:27:43.658343+0000","last_clean":"2023-11-24T06:27:43.658343+0000","last_became_active":"2023-11-24T06:27:42.390011+0000","last_became_peered":"2023-11-24T06:27:42.390011+0000","last_unstale":"2023-11-24T06:27:43.658343+0000","last_undegraded":"2023-11-24T06:27:43.658343+0000","last_fullsized":"2023-11-24T06:27:43.658343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355889+0000","last_change":"2023-11-24T06:27:42.397895+0000","last_active":"2023-11-24T06:27:43.355889+0000","last_peered":"2023-11-24T06:27:43.355889+0000","last_clean":"2023-11-24T06:27:43.355889+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355889+0000","last_undegraded":"2023-11-24T06:27:43.355889+0000","last_fullsized":"2023-11-24T06:27:43.355889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657379+0000","last_change":"2023-11-24T06:27:42.398329+0000","last_active":"2023-11-24T06:27:43.657379+0000","last_peered":"2023-11-24T06:27:43.657379+0000","last_clean":"2023-11-24T06:27:43.657379+0000","last_became_active":"2023-11-24T06:27:42.398095+0000","last_became_peered":"2023-11-24T06:27:42.398095+0000","last_unstale":"2023-11-24T06:27:43.657379+0000","last_undegraded":"2023-11-24T06:27:43.657379+0000","last_fullsized":"2023-11-24T06:27:43.657379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355033+0000","last_change":"2023-11-24T06:27:42.395940+0000","last_active":"2023-11-24T06:27:43.355033+0000","last_peered":"2023-11-24T06:27:43.355033+0000","last_clean":"2023-11-24T06:27:43.355033+0000","last_became_active":"2023-11-24T06:27:42.395595+0000","last_became_peered":"2023-11-24T06:27:42.395595+0000","last_unstale":"2023-11-24T06:27:43.355033+0000","last_undegraded":"2023-11-24T06:27:43.355033+0000","last_fullsized":"2023-11-24T06:27:43.355033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659266+0000","last_change":"2023-11-24T06:27:42.394418+0000","last_active":"2023-11-24T06:27:43.659266+0000","last_peered":"2023-11-24T06:27:43.659266+0000","last_clean":"2023-11-24T06:27:43.659266+0000","last_became_active":"2023-11-24T06:27:42.394103+0000","last_became_peered":"2023-11-24T06:27:42.394103+0000","last_unstale":"2023-11-24T06:27:43.659266+0000","last_undegraded":"2023-11-24T06:27:43.659266+0000","last_fullsized":"2023-11-24T06:27:43.659266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658516+0000","last_change":"2023-11-24T06:27:42.386091+0000","last_active":"2023-11-24T06:27:43.658516+0000","last_peered":"2023-11-24T06:27:43.658516+0000","last_clean":"2023-11-24T06:27:43.658516+0000","last_became_active":"2023-11-24T06:27:42.385638+0000","last_became_peered":"2023-11-24T06:27:42.385638+0000","last_unstale":"2023-11-24T06:27:43.658516+0000","last_undegraded":"2023-11-24T06:27:43.658516+0000","last_fullsized":"2023-11-24T06:27:43.658516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657666+0000","last_change":"2023-11-24T06:27:42.385847+0000","last_active":"2023-11-24T06:27:43.657666+0000","last_peered":"2023-11-24T06:27:43.657666+0000","last_clean":"2023-11-24T06:27:43.657666+0000","last_became_active":"2023-11-24T06:27:42.384548+0000","last_became_peered":"2023-11-24T06:27:42.384548+0000","last_unstale":"2023-11-24T06:27:43.657666+0000","last_undegraded":"2023-11-24T06:27:43.657666+0000","last_fullsized":"2023-11-24T06:27:43.657666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359608+0000","last_change":"2023-11-24T06:27:42.381135+0000","last_active":"2023-11-24T06:27:43.359608+0000","last_peered":"2023-11-24T06:27:43.359608+0000","last_clean":"2023-11-24T06:27:43.359608+0000","last_became_active":"2023-11-24T06:27:42.379951+0000","last_became_peered":"2023-11-24T06:27:42.379951+0000","last_unstale":"2023-11-24T06:27:43.359608+0000","last_undegraded":"2023-11-24T06:27:43.359608+0000","last_fullsized":"2023-11-24T06:27:43.359608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355537+0000","last_change":"2023-11-24T06:27:42.396365+0000","last_active":"2023-11-24T06:27:43.355537+0000","last_peered":"2023-11-24T06:27:43.355537+0000","last_clean":"2023-11-24T06:27:43.355537+0000","last_became_active":"2023-11-24T06:27:42.394255+0000","last_became_peered":"2023-11-24T06:27:42.394255+0000","last_unstale":"2023-11-24T06:27:43.355537+0000","last_undegraded":"2023-11-24T06:27:43.355537+0000","last_fullsized":"2023-11-24T06:27:43.355537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660061+0000","last_change":"2023-11-24T06:27:42.385401+0000","last_active":"2023-11-24T06:27:43.660061+0000","last_peered":"2023-11-24T06:27:43.660061+0000","last_clean":"2023-11-24T06:27:43.660061+0000","last_became_active":"2023-11-24T06:27:42.384001+0000","last_became_peered":"2023-11-24T06:27:42.384001+0000","last_unstale":"2023-11-24T06:27:43.660061+0000","last_undegraded":"2023-11-24T06:27:43.660061+0000","last_fullsized":"2023-11-24T06:27:43.660061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:50.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657883+0000","last_change":"2023-11-24T06:27:42.375262+0000","last_active":"2023-11-24T06:27:43.657883+0000","last_peered":"2023-11-24T06:27:43.657883+0000","last_clean":"2023-11-24T06:27:43.657883+0000","last_became_active":"2023-11-24T06:27:42.374748+0000","last_became_peered":"2023-11-24T06:27:42.374748+0000","last_unstale":"2023-11-24T06:27:43.657883+0000","last_undegraded":"2023-11-24T06:27:43.657883+0000","last_fullsized":"2023-11-24T06:27:43.657883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355370+0000","last_change":"2023-11-24T06:27:42.397612+0000","last_active":"2023-11-24T06:27:43.355370+0000","last_peered":"2023-11-24T06:27:43.355370+0000","last_clean":"2023-11-24T06:27:43.355370+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355370+0000","last_undegraded":"2023-11-24T06:27:43.355370+0000","last_fullsized":"2023-11-24T06:27:43.355370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359434+0000","last_change":"2023-11-24T06:27:42.388744+0000","last_active":"2023-11-24T06:27:43.359434+0000","last_peered":"2023-11-24T06:27:43.359434+0000","last_clean":"2023-11-24T06:27:43.359434+0000","last_became_active":"2023-11-24T06:27:42.388298+0000","last_became_peered":"2023-11-24T06:27:42.388298+0000","last_unstale":"2023-11-24T06:27:43.359434+0000","last_undegraded":"2023-11-24T06:27:43.359434+0000","last_fullsized":"2023-11-24T06:27:43.359434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658048+0000","last_change":"2023-11-24T06:27:42.377765+0000","last_active":"2023-11-24T06:27:43.658048+0000","last_peered":"2023-11-24T06:27:43.658048+0000","last_clean":"2023-11-24T06:27:43.658048+0000","last_became_active":"2023-11-24T06:27:42.377032+0000","last_became_peered":"2023-11-24T06:27:42.377032+0000","last_unstale":"2023-11-24T06:27:43.658048+0000","last_undegraded":"2023-11-24T06:27:43.658048+0000","last_fullsized":"2023-11-24T06:27:43.658048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658694+0000","last_change":"2023-11-24T06:27:42.390624+0000","last_active":"2023-11-24T06:27:43.658694+0000","last_peered":"2023-11-24T06:27:43.658694+0000","last_clean":"2023-11-24T06:27:43.658694+0000","last_became_active":"2023-11-24T06:27:42.387611+0000","last_became_peered":"2023-11-24T06:27:42.387611+0000","last_unstale":"2023-11-24T06:27:43.658694+0000","last_undegraded":"2023-11-24T06:27:43.658694+0000","last_fullsized":"2023-11-24T06:27:43.658694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658079+0000","last_change":"2023-11-24T06:27:42.384846+0000","last_active":"2023-11-24T06:27:43.658079+0000","last_peered":"2023-11-24T06:27:43.658079+0000","last_clean":"2023-11-24T06:27:43.658079+0000","last_became_active":"2023-11-24T06:27:42.384575+0000","last_became_peered":"2023-11-24T06:27:42.384575+0000","last_unstale":"2023-11-24T06:27:43.658079+0000","last_undegraded":"2023-11-24T06:27:43.658079+0000","last_fullsized":"2023-11-24T06:27:43.658079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660460+0000","last_change":"2023-11-24T06:27:42.393160+0000","last_active":"2023-11-24T06:27:43.660460+0000","last_peered":"2023-11-24T06:27:43.660460+0000","last_clean":"2023-11-24T06:27:43.660460+0000","last_became_active":"2023-11-24T06:27:42.392277+0000","last_became_peered":"2023-11-24T06:27:42.392277+0000","last_unstale":"2023-11-24T06:27:43.660460+0000","last_undegraded":"2023-11-24T06:27:43.660460+0000","last_fullsized":"2023-11-24T06:27:43.660460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355922+0000","last_change":"2023-11-24T06:27:42.390004+0000","last_active":"2023-11-24T06:27:43.355922+0000","last_peered":"2023-11-24T06:27:43.355922+0000","last_clean":"2023-11-24T06:27:43.355922+0000","last_became_active":"2023-11-24T06:27:42.389682+0000","last_became_peered":"2023-11-24T06:27:42.389682+0000","last_unstale":"2023-11-24T06:27:43.355922+0000","last_undegraded":"2023-11-24T06:27:43.355922+0000","last_fullsized":"2023-11-24T06:27:43.355922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355110+0000","last_change":"2023-11-24T06:27:42.380022+0000","last_active":"2023-11-24T06:27:43.355110+0000","last_peered":"2023-11-24T06:27:43.355110+0000","last_clean":"2023-11-24T06:27:43.355110+0000","last_became_active":"2023-11-24T06:27:42.379510+0000","last_became_peered":"2023-11-24T06:27:42.379510+0000","last_unstale":"2023-11-24T06:27:43.355110+0000","last_undegraded":"2023-11-24T06:27:43.355110+0000","last_fullsized":"2023-11-24T06:27:43.355110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355598+0000","last_change":"2023-11-24T06:27:42.392966+0000","last_active":"2023-11-24T06:27:43.355598+0000","last_peered":"2023-11-24T06:27:43.355598+0000","last_clean":"2023-11-24T06:27:43.355598+0000","last_became_active":"2023-11-24T06:27:42.392683+0000","last_became_peered":"2023-11-24T06:27:42.392683+0000","last_unstale":"2023-11-24T06:27:43.355598+0000","last_undegraded":"2023-11-24T06:27:43.355598+0000","last_fullsized":"2023-11-24T06:27:43.355598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356153+0000","last_change":"2023-11-24T06:27:42.384496+0000","last_active":"2023-11-24T06:27:43.356153+0000","last_peered":"2023-11-24T06:27:43.356153+0000","last_clean":"2023-11-24T06:27:43.356153+0000","last_became_active":"2023-11-24T06:27:42.384121+0000","last_became_peered":"2023-11-24T06:27:42.384121+0000","last_unstale":"2023-11-24T06:27:43.356153+0000","last_undegraded":"2023-11-24T06:27:43.356153+0000","last_fullsized":"2023-11-24T06:27:43.356153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:09.683680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658776+0000","last_change":"2023-11-24T06:27:42.386718+0000","last_active":"2023-11-24T06:27:43.658776+0000","last_peered":"2023-11-24T06:27:43.658776+0000","last_clean":"2023-11-24T06:27:43.658776+0000","last_became_active":"2023-11-24T06:27:42.386397+0000","last_became_peered":"2023-11-24T06:27:42.386397+0000","last_unstale":"2023-11-24T06:27:43.658776+0000","last_undegraded":"2023-11-24T06:27:43.658776+0000","last_fullsized":"2023-11-24T06:27:43.658776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658422+0000","last_change":"2023-11-24T06:27:42.389793+0000","last_active":"2023-11-24T06:27:43.658422+0000","last_peered":"2023-11-24T06:27:43.658422+0000","last_clean":"2023-11-24T06:27:43.658422+0000","last_became_active":"2023-11-24T06:27:42.389493+0000","last_became_peered":"2023-11-24T06:27:42.389493+0000","last_unstale":"2023-11-24T06:27:43.658422+0000","last_undegraded":"2023-11-24T06:27:43.658422+0000","last_fullsized":"2023-11-24T06:27:43.658422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356062+0000","last_change":"2023-11-24T06:27:42.389497+0000","last_active":"2023-11-24T06:27:43.356062+0000","last_peered":"2023-11-24T06:27:43.356062+0000","last_clean":"2023-11-24T06:27:43.356062+0000","last_became_active":"2023-11-24T06:27:42.389132+0000","last_became_peered":"2023-11-24T06:27:42.389132+0000","last_unstale":"2023-11-24T06:27:43.356062+0000","last_undegraded":"2023-11-24T06:27:43.356062+0000","last_fullsized":"2023-11-24T06:27:43.356062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:37.697060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660457+0000","last_change":"2023-11-24T06:27:42.398466+0000","last_active":"2023-11-24T06:27:43.660457+0000","last_peered":"2023-11-24T06:27:43.660457+0000","last_clean":"2023-11-24T06:27:43.660457+0000","last_became_active":"2023-11-24T06:27:42.397061+0000","last_became_peered":"2023-11-24T06:27:42.397061+0000","last_unstale":"2023-11-24T06:27:43.660457+0000","last_undegraded":"2023-11-24T06:27:43.660457+0000","last_fullsized":"2023-11-24T06:27:43.660457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356264+0000","last_change":"2023-11-24T06:27:42.385278+0000","last_active":"2023-11-24T06:27:43.356264+0000","last_peered":"2023-11-24T06:27:43.356264+0000","last_clean":"2023-11-24T06:27:43.356264+0000","last_became_active":"2023-11-24T06:27:42.384909+0000","last_became_peered":"2023-11-24T06:27:42.384909+0000","last_unstale":"2023-11-24T06:27:43.356264+0000","last_undegraded":"2023-11-24T06:27:43.356264+0000","last_fullsized":"2023-11-24T06:27:43.356264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656675+0000","last_change":"2023-11-24T06:27:42.382417+0000","last_active":"2023-11-24T06:27:43.656675+0000","last_peered":"2023-11-24T06:27:43.656675+0000","last_clean":"2023-11-24T06:27:43.656675+0000","last_became_active":"2023-11-24T06:27:42.381985+0000","last_became_peered":"2023-11-24T06:27:42.381985+0000","last_unstale":"2023-11-24T06:27:43.656675+0000","last_undegraded":"2023-11-24T06:27:43.656675+0000","last_fullsized":"2023-11-24T06:27:43.656675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656964+0000","last_change":"2023-11-24T06:27:42.395403+0000","last_active":"2023-11-24T06:27:43.656964+0000","last_peered":"2023-11-24T06:27:43.656964+0000","last_clean":"2023-11-24T06:27:43.656964+0000","last_became_active":"2023-11-24T06:27:42.394442+0000","last_became_peered":"2023-11-24T06:27:42.394442+0000","last_unstale":"2023-11-24T06:27:43.656964+0000","last_undegraded":"2023-11-24T06:27:43.656964+0000","last_fullsized":"2023-11-24T06:27:43.656964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357092+0000","last_change":"2023-11-24T06:27:42.375948+0000","last_active":"2023-11-24T06:27:43.357092+0000","last_peered":"2023-11-24T06:27:43.357092+0000","last_clean":"2023-11-24T06:27:43.357092+0000","last_became_active":"2023-11-24T06:27:42.374443+0000","last_became_peered":"2023-11-24T06:27:42.374443+0000","last_unstale":"2023-11-24T06:27:43.357092+0000","last_undegraded":"2023-11-24T06:27:43.357092+0000","last_fullsized":"2023-11-24T06:27:43.357092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:11.141034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355607+0000","last_change":"2023-11-24T06:27:41.381254+0000","last_active":"2023-11-24T06:27:43.355607+0000","last_peered":"2023-11-24T06:27:43.355607+0000","last_clean":"2023-11-24T06:27:43.355607+0000","last_became_active":"2023-11-24T06:27:41.380768+0000","last_became_peered":"2023-11-24T06:27:41.380768+0000","last_unstale":"2023-11-24T06:27:43.355607+0000","last_undegraded":"2023-11-24T06:27:43.355607+0000","last_fullsized":"2023-11-24T06:27:43.355607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660887+0000","last_change":"2023-11-24T06:27:42.393018+0000","last_active":"2023-11-24T06:27:43.660887+0000","last_peered":"2023-11-24T06:27:43.660887+0000","last_clean":"2023-11-24T06:27:43.660887+0000","last_became_active":"2023-11-24T06:27:42.391726+0000","last_became_peered":"2023-11-24T06:27:42.391726+0000","last_unstale":"2023-11-24T06:27:43.660887+0000","last_undegraded":"2023-11-24T06:27:43.660887+0000","last_fullsized":"2023-11-24T06:27:43.660887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657580+0000","last_change":"2023-11-24T06:27:41.383020+0000","last_active":"2023-11-24T06:27:43.657580+0000","last_peered":"2023-11-24T06:27:43.657580+0000","last_clean":"2023-11-24T06:27:43.657580+0000","last_became_active":"2023-11-24T06:27:41.382642+0000","last_became_peered":"2023-11-24T06:27:41.382642+0000","last_unstale":"2023-11-24T06:27:43.657580+0000","last_undegraded":"2023-11-24T06:27:43.657580+0000","last_fullsized":"2023-11-24T06:27:43.657580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:19.408882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355377+0000","last_change":"2023-11-24T06:27:41.382979+0000","last_active":"2023-11-24T06:27:43.355377+0000","last_peered":"2023-11-24T06:27:43.355377+0000","last_clean":"2023-11-24T06:27:43.355377+0000","last_became_active":"2023-11-24T06:27:41.381955+0000","last_became_peered":"2023-11-24T06:27:41.381955+0000","last_unstale":"2023-11-24T06:27:43.355377+0000","last_undegraded":"2023-11-24T06:27:43.355377+0000","last_fullsized":"2023-11-24T06:27:43.355377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.354996+0000","last_change":"2023-11-24T06:27:42.395234+0000","last_active":"2023-11-24T06:27:43.354996+0000","last_peered":"2023-11-24T06:27:43.354996+0000","last_clean":"2023-11-24T06:27:43.354996+0000","last_became_active":"2023-11-24T06:27:42.394890+0000","last_became_peered":"2023-11-24T06:27:42.394890+0000","last_unstale":"2023-11-24T06:27:43.354996+0000","last_undegraded":"2023-11-24T06:27:43.354996+0000","last_fullsized":"2023-11-24T06:27:43.354996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355626+0000","last_change":"2023-11-24T06:27:41.371070+0000","last_active":"2023-11-24T06:27:43.355626+0000","last_peered":"2023-11-24T06:27:43.355626+0000","last_clean":"2023-11-24T06:27:43.355626+0000","last_became_active":"2023-11-24T06:27:41.368037+0000","last_became_peered":"2023-11-24T06:27:41.368037+0000","last_unstale":"2023-11-24T06:27:43.355626+0000","last_undegraded":"2023-11-24T06:27:43.355626+0000","last_fullsized":"2023-11-24T06:27:43.355626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657673+0000","last_change":"2023-11-24T06:27:42.385239+0000","last_active":"2023-11-24T06:27:43.657673+0000","last_peered":"2023-11-24T06:27:43.657673+0000","last_clean":"2023-11-24T06:27:43.657673+0000","last_became_active":"2023-11-24T06:27:42.384436+0000","last_became_peered":"2023-11-24T06:27:42.384436+0000","last_unstale":"2023-11-24T06:27:43.657673+0000","last_undegraded":"2023-11-24T06:27:43.657673+0000","last_fullsized":"2023-11-24T06:27:43.657673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657925+0000","last_change":"2023-11-24T06:27:41.381169+0000","last_active":"2023-11-24T06:27:43.657925+0000","last_peered":"2023-11-24T06:27:43.657925+0000","last_clean":"2023-11-24T06:27:43.657925+0000","last_became_active":"2023-11-24T06:27:41.380835+0000","last_became_peered":"2023-11-24T06:27:41.380835+0000","last_unstale":"2023-11-24T06:27:43.657925+0000","last_undegraded":"2023-11-24T06:27:43.657925+0000","last_fullsized":"2023-11-24T06:27:43.657925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358731+0000","last_change":"2023-11-24T06:27:42.390319+0000","last_active":"2023-11-24T06:27:43.358731+0000","last_peered":"2023-11-24T06:27:43.358731+0000","last_clean":"2023-11-24T06:27:43.358731+0000","last_became_active":"2023-11-24T06:27:42.388441+0000","last_became_peered":"2023-11-24T06:27:42.388441+0000","last_unstale":"2023-11-24T06:27:43.358731+0000","last_undegraded":"2023-11-24T06:27:43.358731+0000","last_fullsized":"2023-11-24T06:27:43.358731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358844+0000","last_change":"2023-11-24T06:27:41.385916+0000","last_active":"2023-11-24T06:27:43.358844+0000","last_peered":"2023-11-24T06:27:43.358844+0000","last_clean":"2023-11-24T06:27:43.358844+0000","last_became_active":"2023-11-24T06:27:41.385609+0000","last_became_peered":"2023-11-24T06:27:41.385609+0000","last_unstale":"2023-11-24T06:27:43.358844+0000","last_undegraded":"2023-11-24T06:27:43.358844+0000","last_fullsized":"2023-11-24T06:27:43.358844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660892+0000","last_change":"2023-11-24T06:27:42.395957+0000","last_active":"2023-11-24T06:27:43.660892+0000","last_peered":"2023-11-24T06:27:43.660892+0000","last_clean":"2023-11-24T06:27:43.660892+0000","last_became_active":"2023-11-24T06:27:42.395222+0000","last_became_peered":"2023-11-24T06:27:42.395222+0000","last_unstale":"2023-11-24T06:27:43.660892+0000","last_undegraded":"2023-11-24T06:27:43.660892+0000","last_fullsized":"2023-11-24T06:27:43.660892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658670+0000","last_change":"2023-11-24T06:27:41.382629+0000","last_active":"2023-11-24T06:27:43.658670+0000","last_peered":"2023-11-24T06:27:43.658670+0000","last_clean":"2023-11-24T06:27:43.658670+0000","last_became_active":"2023-11-24T06:27:41.382323+0000","last_became_peered":"2023-11-24T06:27:41.382323+0000","last_unstale":"2023-11-24T06:27:43.658670+0000","last_undegraded":"2023-11-24T06:27:43.658670+0000","last_fullsized":"2023-11-24T06:27:43.658670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:38.362359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359838+0000","last_change":"2023-11-24T06:27:42.373431+0000","last_active":"2023-11-24T06:27:43.359838+0000","last_peered":"2023-11-24T06:27:43.359838+0000","last_clean":"2023-11-24T06:27:43.359838+0000","last_became_active":"2023-11-24T06:27:42.373106+0000","last_became_peered":"2023-11-24T06:27:42.373106+0000","last_unstale":"2023-11-24T06:27:43.359838+0000","last_undegraded":"2023-11-24T06:27:43.359838+0000","last_fullsized":"2023-11-24T06:27:43.359838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658482+0000","last_change":"2023-11-24T06:27:41.381556+0000","last_active":"2023-11-24T06:27:43.658482+0000","last_peered":"2023-11-24T06:27:43.658482+0000","last_clean":"2023-11-24T06:27:43.658482+0000","last_became_active":"2023-11-24T06:27:41.380552+0000","last_became_peered":"2023-11-24T06:27:41.380552+0000","last_unstale":"2023-11-24T06:27:43.658482+0000","last_undegraded":"2023-11-24T06:27:43.658482+0000","last_fullsized":"2023-11-24T06:27:43.658482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:48.968587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659001+0000","last_change":"2023-11-24T06:27:42.391222+0000","last_active":"2023-11-24T06:27:43.659001+0000","last_peered":"2023-11-24T06:27:43.659001+0000","last_clean":"2023-11-24T06:27:43.659001+0000","last_became_active":"2023-11-24T06:27:42.390910+0000","last_became_peered":"2023-11-24T06:27:42.390910+0000","last_unstale":"2023-11-24T06:27:43.659001+0000","last_undegraded":"2023-11-24T06:27:43.659001+0000","last_fullsized":"2023-11-24T06:27:43.659001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660728+0000","last_change":"2023-11-24T06:27:41.374697+0000","last_active":"2023-11-24T06:27:43.660728+0000","last_peered":"2023-11-24T06:27:43.660728+0000","last_clean":"2023-11-24T06:27:43.660728+0000","last_became_active":"2023-11-24T06:27:41.374419+0000","last_became_peered":"2023-11-24T06:27:41.374419+0000","last_unstale":"2023-11-24T06:27:43.660728+0000","last_undegraded":"2023-11-24T06:27:43.660728+0000","last_fullsized":"2023-11-24T06:27:43.660728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660079+0000","last_change":"2023-11-24T06:27:42.393174+0000","last_active":"2023-11-24T06:27:43.660079+0000","last_peered":"2023-11-24T06:27:43.660079+0000","last_clean":"2023-11-24T06:27:43.660079+0000","last_became_active":"2023-11-24T06:27:42.392964+0000","last_became_peered":"2023-11-24T06:27:42.392964+0000","last_unstale":"2023-11-24T06:27:43.660079+0000","last_undegraded":"2023-11-24T06:27:43.660079+0000","last_fullsized":"2023-11-24T06:27:43.660079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356229+0000","last_change":"2023-11-24T06:27:41.379781+0000","last_active":"2023-11-24T06:27:43.356229+0000","last_peered":"2023-11-24T06:27:43.356229+0000","last_clean":"2023-11-24T06:27:43.356229+0000","last_became_active":"2023-11-24T06:27:41.379484+0000","last_became_peered":"2023-11-24T06:27:41.379484+0000","last_unstale":"2023-11-24T06:27:43.356229+0000","last_undegraded":"2023-11-24T06:27:43.356229+0000","last_fullsized":"2023-11-24T06:27:43.356229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658712+0000","last_change":"2023-11-24T06:27:42.385192+0000","last_active":"2023-11-24T06:27:43.658712+0000","last_peered":"2023-11-24T06:27:43.658712+0000","last_clean":"2023-11-24T06:27:43.658712+0000","last_became_active":"2023-11-24T06:27:42.384777+0000","last_became_peered":"2023-11-24T06:27:42.384777+0000","last_unstale":"2023-11-24T06:27:43.658712+0000","last_undegraded":"2023-11-24T06:27:43.658712+0000","last_fullsized":"2023-11-24T06:27:43.658712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657173+0000","last_change":"2023-11-24T06:27:41.383717+0000","last_active":"2023-11-24T06:27:43.657173+0000","last_peered":"2023-11-24T06:27:43.657173+0000","last_clean":"2023-11-24T06:27:43.657173+0000","last_became_active":"2023-11-24T06:27:41.382246+0000","last_became_peered":"2023-11-24T06:27:41.382246+0000","last_unstale":"2023-11-24T06:27:43.657173+0000","last_undegraded":"2023-11-24T06:27:43.657173+0000","last_fullsized":"2023-11-24T06:27:43.657173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:04.357908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656917+0000","last_change":"2023-11-24T06:27:42.392505+0000","last_active":"2023-11-24T06:27:43.656917+0000","last_peered":"2023-11-24T06:27:43.656917+0000","last_clean":"2023-11-24T06:27:43.656917+0000","last_became_active":"2023-11-24T06:27:42.391756+0000","last_became_peered":"2023-11-24T06:27:42.391756+0000","last_unstale":"2023-11-24T06:27:43.656917+0000","last_undegraded":"2023-11-24T06:27:43.656917+0000","last_fullsized":"2023-11-24T06:27:43.656917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656383+0000","last_change":"2023-11-24T06:27:41.366927+0000","last_active":"2023-11-24T06:27:43.656383+0000","last_peered":"2023-11-24T06:27:43.656383+0000","last_clean":"2023-11-24T06:27:43.656383+0000","last_became_active":"2023-11-24T06:27:41.366457+0000","last_became_peered":"2023-11-24T06:27:41.366457+0000","last_unstale":"2023-11-24T06:27:43.656383+0000","last_undegraded":"2023-11-24T06:27:43.656383+0000","last_fullsized":"2023-11-24T06:27:43.656383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657400+0000","last_change":"2023-11-24T06:27:42.386516+0000","last_active":"2023-11-24T06:27:43.657400+0000","last_peered":"2023-11-24T06:27:43.657400+0000","last_clean":"2023-11-24T06:27:43.657400+0000","last_became_active":"2023-11-24T06:27:42.386236+0000","last_became_peered":"2023-11-24T06:27:42.386236+0000","last_unstale":"2023-11-24T06:27:43.657400+0000","last_undegraded":"2023-11-24T06:27:43.657400+0000","last_fullsized":"2023-11-24T06:27:43.657400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358955+0000","last_change":"2023-11-24T06:27:41.368887+0000","last_active":"2023-11-24T06:27:43.358955+0000","last_peered":"2023-11-24T06:27:43.358955+0000","last_clean":"2023-11-24T06:27:43.358955+0000","last_became_active":"2023-11-24T06:27:41.367135+0000","last_became_peered":"2023-11-24T06:27:41.367135+0000","last_unstale":"2023-11-24T06:27:43.358955+0000","last_undegraded":"2023-11-24T06:27:43.358955+0000","last_fullsized":"2023-11-24T06:27:43.358955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:11.584532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659288+0000","last_change":"2023-11-24T06:27:42.391675+0000","last_active":"2023-11-24T06:27:43.659288+0000","last_peered":"2023-11-24T06:27:43.659288+0000","last_clean":"2023-11-24T06:27:43.659288+0000","last_became_active":"2023-11-24T06:27:42.391212+0000","last_became_peered":"2023-11-24T06:27:42.391212+0000","last_unstale":"2023-11-24T06:27:43.659288+0000","last_undegraded":"2023-11-24T06:27:43.659288+0000","last_fullsized":"2023-11-24T06:27:43.659288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658283+0000","last_change":"2023-11-24T06:27:41.377474+0000","last_active":"2023-11-24T06:27:43.658283+0000","last_peered":"2023-11-24T06:27:43.658283+0000","last_clean":"2023-11-24T06:27:43.658283+0000","last_became_active":"2023-11-24T06:27:41.377079+0000","last_became_peered":"2023-11-24T06:27:41.377079+0000","last_unstale":"2023-11-24T06:27:43.658283+0000","last_undegraded":"2023-11-24T06:27:43.658283+0000","last_fullsized":"2023-11-24T06:27:43.658283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659627+0000","last_change":"2023-11-24T06:27:42.388823+0000","last_active":"2023-11-24T06:27:43.659627+0000","last_peered":"2023-11-24T06:27:43.659627+0000","last_clean":"2023-11-24T06:27:43.659627+0000","last_became_active":"2023-11-24T06:27:42.388562+0000","last_became_peered":"2023-11-24T06:27:42.388562+0000","last_unstale":"2023-11-24T06:27:43.659627+0000","last_undegraded":"2023-11-24T06:27:43.659627+0000","last_fullsized":"2023-11-24T06:27:43.659627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:02.353531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658740+0000","last_change":"2023-11-24T06:27:41.375328+0000","last_active":"2023-11-24T06:27:43.658740+0000","last_peered":"2023-11-24T06:27:43.658740+0000","last_clean":"2023-11-24T06:27:43.658740+0000","last_became_active":"2023-11-24T06:27:41.375049+0000","last_became_peered":"2023-11-24T06:27:41.375049+0000","last_unstale":"2023-11-24T06:27:43.658740+0000","last_undegraded":"2023-11-24T06:27:43.658740+0000","last_fullsized":"2023-11-24T06:27:43.658740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358153+0000","last_change":"2023-11-24T06:27:42.398402+0000","last_active":"2023-11-24T06:27:43.358153+0000","last_peered":"2023-11-24T06:27:43.358153+0000","last_clean":"2023-11-24T06:27:43.358153+0000","last_became_active":"2023-11-24T06:27:42.393706+0000","last_became_peered":"2023-11-24T06:27:42.393706+0000","last_unstale":"2023-11-24T06:27:43.358153+0000","last_undegraded":"2023-11-24T06:27:43.358153+0000","last_fullsized":"2023-11-24T06:27:43.358153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657227+0000","last_change":"2023-11-24T06:27:41.366299+0000","last_active":"2023-11-24T06:27:43.657227+0000","last_peered":"2023-11-24T06:27:43.657227+0000","last_clean":"2023-11-24T06:27:43.657227+0000","last_became_active":"2023-11-24T06:27:41.365987+0000","last_became_peered":"2023-11-24T06:27:41.365987+0000","last_unstale":"2023-11-24T06:27:43.657227+0000","last_undegraded":"2023-11-24T06:27:43.657227+0000","last_fullsized":"2023-11-24T06:27:43.657227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:31.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658289+0000","last_change":"2023-11-24T06:27:42.392311+0000","last_active":"2023-11-24T06:27:43.658289+0000","last_peered":"2023-11-24T06:27:43.658289+0000","last_clean":"2023-11-24T06:27:43.658289+0000","last_became_active":"2023-11-24T06:27:42.391517+0000","last_became_peered":"2023-11-24T06:27:42.391517+0000","last_unstale":"2023-11-24T06:27:43.658289+0000","last_undegraded":"2023-11-24T06:27:43.658289+0000","last_fullsized":"2023-11-24T06:27:43.658289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.372912+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:27:42.372912+0000","last_peered":"2023-11-24T06:27:42.372912+0000","last_clean":"2023-11-24T06:27:42.372912+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:27:42.372912+0000","last_undegraded":"2023-11-24T06:27:42.372912+0000","last_fullsized":"2023-11-24T06:27:42.372912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355438+0000","last_change":"2023-11-24T06:27:41.370135+0000","last_active":"2023-11-24T06:27:43.355438+0000","last_peered":"2023-11-24T06:27:43.355438+0000","last_clean":"2023-11-24T06:27:43.355438+0000","last_became_active":"2023-11-24T06:27:41.369372+0000","last_became_peered":"2023-11-24T06:27:41.369372+0000","last_unstale":"2023-11-24T06:27:43.355438+0000","last_undegraded":"2023-11-24T06:27:43.355438+0000","last_fullsized":"2023-11-24T06:27:43.355438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:07.305138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355221+0000","last_change":"2023-11-24T06:27:42.384732+0000","last_active":"2023-11-24T06:27:43.355221+0000","last_peered":"2023-11-24T06:27:43.355221+0000","last_clean":"2023-11-24T06:27:43.355221+0000","last_became_active":"2023-11-24T06:27:42.384083+0000","last_became_peered":"2023-11-24T06:27:42.384083+0000","last_unstale":"2023-11-24T06:27:43.355221+0000","last_undegraded":"2023-11-24T06:27:43.355221+0000","last_fullsized":"2023-11-24T06:27:43.355221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656745+0000","last_change":"2023-11-24T06:27:41.384707+0000","last_active":"2023-11-24T06:27:43.656745+0000","last_peered":"2023-11-24T06:27:43.656745+0000","last_clean":"2023-11-24T06:27:43.656745+0000","last_became_active":"2023-11-24T06:27:41.384397+0000","last_became_peered":"2023-11-24T06:27:41.384397+0000","last_unstale":"2023-11-24T06:27:43.656745+0000","last_undegraded":"2023-11-24T06:27:43.656745+0000","last_fullsized":"2023-11-24T06:27:43.656745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659143+0000","last_change":"2023-11-24T06:27:42.394007+0000","last_active":"2023-11-24T06:27:43.659143+0000","last_peered":"2023-11-24T06:27:43.659143+0000","last_clean":"2023-11-24T06:27:43.659143+0000","last_became_active":"2023-11-24T06:27:42.393636+0000","last_became_peered":"2023-11-24T06:27:42.393636+0000","last_unstale":"2023-11-24T06:27:43.659143+0000","last_undegraded":"2023-11-24T06:27:43.659143+0000","last_fullsized":"2023-11-24T06:27:43.659143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:05.800693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656767+0000","last_change":"2023-11-24T06:27:41.375368+0000","last_active":"2023-11-24T06:27:43.656767+0000","last_peered":"2023-11-24T06:27:43.656767+0000","last_clean":"2023-11-24T06:27:43.656767+0000","last_became_active":"2023-11-24T06:27:41.375062+0000","last_became_peered":"2023-11-24T06:27:41.375062+0000","last_unstale":"2023-11-24T06:27:43.656767+0000","last_undegraded":"2023-11-24T06:27:43.656767+0000","last_fullsized":"2023-11-24T06:27:43.656767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657273+0000","last_change":"2023-11-24T06:27:42.375816+0000","last_active":"2023-11-24T06:27:43.657273+0000","last_peered":"2023-11-24T06:27:43.657273+0000","last_clean":"2023-11-24T06:27:43.657273+0000","last_became_active":"2023-11-24T06:27:42.375209+0000","last_became_peered":"2023-11-24T06:27:42.375209+0000","last_unstale":"2023-11-24T06:27:43.657273+0000","last_undegraded":"2023-11-24T06:27:43.657273+0000","last_fullsized":"2023-11-24T06:27:43.657273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356462+0000","last_change":"2023-11-24T06:27:41.376512+0000","last_active":"2023-11-24T06:27:43.356462+0000","last_peered":"2023-11-24T06:27:43.356462+0000","last_clean":"2023-11-24T06:27:43.356462+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356462+0000","last_undegraded":"2023-11-24T06:27:43.356462+0000","last_fullsized":"2023-11-24T06:27:43.356462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358784+0000","last_change":"2023-11-24T06:27:42.395282+0000","last_active":"2023-11-24T06:27:43.358784+0000","last_peered":"2023-11-24T06:27:43.358784+0000","last_clean":"2023-11-24T06:27:43.358784+0000","last_became_active":"2023-11-24T06:27:42.394947+0000","last_became_peered":"2023-11-24T06:27:42.394947+0000","last_unstale":"2023-11-24T06:27:43.358784+0000","last_undegraded":"2023-11-24T06:27:43.358784+0000","last_fullsized":"2023-11-24T06:27:43.358784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:03.529167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658293+0000","last_change":"2023-11-24T06:27:41.377691+0000","last_active":"2023-11-24T06:27:43.658293+0000","last_peered":"2023-11-24T06:27:43.658293+0000","last_clean":"2023-11-24T06:27:43.658293+0000","last_became_active":"2023-11-24T06:27:41.377407+0000","last_became_peered":"2023-11-24T06:27:41.377407+0000","last_unstale":"2023-11-24T06:27:43.658293+0000","last_undegraded":"2023-11-24T06:27:43.658293+0000","last_fullsized":"2023-11-24T06:27:43.658293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:31.799834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358354+0000","last_change":"2023-11-24T06:27:42.388531+0000","last_active":"2023-11-24T06:27:43.358354+0000","last_peered":"2023-11-24T06:27:43.358354+0000","last_clean":"2023-11-24T06:27:43.358354+0000","last_became_active":"2023-11-24T06:27:42.388137+0000","last_became_peered":"2023-11-24T06:27:42.388137+0000","last_unstale":"2023-11-24T06:27:43.358354+0000","last_undegraded":"2023-11-24T06:27:43.358354+0000","last_fullsized":"2023-11-24T06:27:43.358354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658823+0000","last_change":"2023-11-24T06:27:41.383929+0000","last_active":"2023-11-24T06:27:43.658823+0000","last_peered":"2023-11-24T06:27:43.658823+0000","last_clean":"2023-11-24T06:27:43.658823+0000","last_became_active":"2023-11-24T06:27:41.383652+0000","last_became_peered":"2023-11-24T06:27:41.383652+0000","last_unstale":"2023-11-24T06:27:43.658823+0000","last_undegraded":"2023-11-24T06:27:43.658823+0000","last_fullsized":"2023-11-24T06:27:43.658823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658415+0000","last_change":"2023-11-24T06:27:42.378135+0000","last_active":"2023-11-24T06:27:43.658415+0000","last_peered":"2023-11-24T06:27:43.658415+0000","last_clean":"2023-11-24T06:27:43.658415+0000","last_became_active":"2023-11-24T06:27:42.377665+0000","last_became_peered":"2023-11-24T06:27:42.377665+0000","last_unstale":"2023-11-24T06:27:43.658415+0000","last_undegraded":"2023-11-24T06:27:43.658415+0000","last_fullsized":"2023-11-24T06:27:43.658415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659071+0000","last_change":"2023-11-24T06:27:41.388698+0000","last_active":"2023-11-24T06:27:43.659071+0000","last_peered":"2023-11-24T06:27:43.659071+0000","last_clean":"2023-11-24T06:27:43.659071+0000","last_became_active":"2023-11-24T06:27:41.388379+0000","last_became_peered":"2023-11-24T06:27:41.388379+0000","last_unstale":"2023-11-24T06:27:43.659071+0000","last_undegraded":"2023-11-24T06:27:43.659071+0000","last_fullsized":"2023-11-24T06:27:43.659071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356256+0000","last_change":"2023-11-24T06:27:42.387047+0000","last_active":"2023-11-24T06:27:43.356256+0000","last_peered":"2023-11-24T06:27:43.356256+0000","last_clean":"2023-11-24T06:27:43.356256+0000","last_became_active":"2023-11-24T06:27:42.386729+0000","last_became_peered":"2023-11-24T06:27:42.386729+0000","last_unstale":"2023-11-24T06:27:43.356256+0000","last_undegraded":"2023-11-24T06:27:43.356256+0000","last_fullsized":"2023-11-24T06:27:43.356256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:31.853326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658188+0000","last_change":"2023-11-24T06:27:41.379713+0000","last_active":"2023-11-24T06:27:43.658188+0000","last_peered":"2023-11-24T06:27:43.658188+0000","last_clean":"2023-11-24T06:27:43.658188+0000","last_became_active":"2023-11-24T06:27:41.379416+0000","last_became_peered":"2023-11-24T06:27:41.379416+0000","last_unstale":"2023-11-24T06:27:43.658188+0000","last_undegraded":"2023-11-24T06:27:43.658188+0000","last_fullsized":"2023-11-24T06:27:43.658188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660748+0000","last_change":"2023-11-24T06:27:42.392358+0000","last_active":"2023-11-24T06:27:43.660748+0000","last_peered":"2023-11-24T06:27:43.660748+0000","last_clean":"2023-11-24T06:27:43.660748+0000","last_became_active":"2023-11-24T06:27:42.392029+0000","last_became_peered":"2023-11-24T06:27:42.392029+0000","last_unstale":"2023-11-24T06:27:43.660748+0000","last_undegraded":"2023-11-24T06:27:43.660748+0000","last_fullsized":"2023-11-24T06:27:43.660748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656706+0000","last_change":"2023-11-24T06:27:41.384873+0000","last_active":"2023-11-24T06:27:43.656706+0000","last_peered":"2023-11-24T06:27:43.656706+0000","last_clean":"2023-11-24T06:27:43.656706+0000","last_became_active":"2023-11-24T06:27:41.384475+0000","last_became_peered":"2023-11-24T06:27:41.384475+0000","last_unstale":"2023-11-24T06:27:43.656706+0000","last_undegraded":"2023-11-24T06:27:43.656706+0000","last_fullsized":"2023-11-24T06:27:43.656706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357526+0000","last_change":"2023-11-24T06:27:42.381195+0000","last_active":"2023-11-24T06:27:43.357526+0000","last_peered":"2023-11-24T06:27:43.357526+0000","last_clean":"2023-11-24T06:27:43.357526+0000","last_became_active":"2023-11-24T06:27:42.377313+0000","last_became_peered":"2023-11-24T06:27:42.377313+0000","last_unstale":"2023-11-24T06:27:43.357526+0000","last_undegraded":"2023-11-24T06:27:43.357526+0000","last_fullsized":"2023-11-24T06:27:43.357526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:32.562262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657763+0000","last_change":"2023-11-24T06:27:41.377860+0000","last_active":"2023-11-24T06:27:43.657763+0000","last_peered":"2023-11-24T06:27:43.657763+0000","last_clean":"2023-11-24T06:27:43.657763+0000","last_became_active":"2023-11-24T06:27:41.377572+0000","last_became_peered":"2023-11-24T06:27:41.377572+0000","last_unstale":"2023-11-24T06:27:43.657763+0000","last_undegraded":"2023-11-24T06:27:43.657763+0000","last_fullsized":"2023-11-24T06:27:43.657763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:52.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355182+0000","last_change":"2023-11-24T06:27:42.385894+0000","last_active":"2023-11-24T06:27:43.355182+0000","last_peered":"2023-11-24T06:27:43.355182+0000","last_clean":"2023-11-24T06:27:43.355182+0000","last_became_active":"2023-11-24T06:27:42.385586+0000","last_became_peered":"2023-11-24T06:27:42.385586+0000","last_unstale":"2023-11-24T06:27:43.355182+0000","last_undegraded":"2023-11-24T06:27:43.355182+0000","last_fullsized":"2023-11-24T06:27:43.355182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658185+0000","last_change":"2023-11-24T06:27:41.380746+0000","last_active":"2023-11-24T06:27:43.658185+0000","last_peered":"2023-11-24T06:27:43.658185+0000","last_clean":"2023-11-24T06:27:43.658185+0000","last_became_active":"2023-11-24T06:27:41.380535+0000","last_became_peered":"2023-11-24T06:27:41.380535+0000","last_unstale":"2023-11-24T06:27:43.658185+0000","last_undegraded":"2023-11-24T06:27:43.658185+0000","last_fullsized":"2023-11-24T06:27:43.658185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:10.313348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355700+0000","last_change":"2023-11-24T06:27:42.399877+0000","last_active":"2023-11-24T06:27:43.355700+0000","last_peered":"2023-11-24T06:27:43.355700+0000","last_clean":"2023-11-24T06:27:43.355700+0000","last_became_active":"2023-11-24T06:27:42.399051+0000","last_became_peered":"2023-11-24T06:27:42.399051+0000","last_unstale":"2023-11-24T06:27:43.355700+0000","last_undegraded":"2023-11-24T06:27:43.355700+0000","last_fullsized":"2023-11-24T06:27:43.355700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:09.167666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659116+0000","last_change":"2023-11-24T06:27:41.386935+0000","last_active":"2023-11-24T06:27:43.659116+0000","last_peered":"2023-11-24T06:27:43.659116+0000","last_clean":"2023-11-24T06:27:43.659116+0000","last_became_active":"2023-11-24T06:27:41.386607+0000","last_became_peered":"2023-11-24T06:27:41.386607+0000","last_unstale":"2023-11-24T06:27:43.659116+0000","last_undegraded":"2023-11-24T06:27:43.659116+0000","last_fullsized":"2023-11-24T06:27:43.659116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659499+0000","last_change":"2023-11-24T06:27:42.382231+0000","last_active":"2023-11-24T06:27:43.659499+0000","last_peered":"2023-11-24T06:27:43.659499+0000","last_clean":"2023-11-24T06:27:43.659499+0000","last_became_active":"2023-11-24T06:27:42.381452+0000","last_became_peered":"2023-11-24T06:27:42.381452+0000","last_unstale":"2023-11-24T06:27:43.659499+0000","last_undegraded":"2023-11-24T06:27:43.659499+0000","last_fullsized":"2023-11-24T06:27:43.659499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358206+0000","last_change":"2023-11-24T06:27:42.378166+0000","last_active":"2023-11-24T06:27:43.358206+0000","last_peered":"2023-11-24T06:27:43.358206+0000","last_clean":"2023-11-24T06:27:43.358206+0000","last_became_active":"2023-11-24T06:27:42.377859+0000","last_became_peered":"2023-11-24T06:27:42.377859+0000","last_unstale":"2023-11-24T06:27:43.358206+0000","last_undegraded":"2023-11-24T06:27:43.358206+0000","last_fullsized":"2023-11-24T06:27:43.358206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657894+0000","last_change":"2023-11-24T06:27:41.386057+0000","last_active":"2023-11-24T06:27:43.657894+0000","last_peered":"2023-11-24T06:27:43.657894+0000","last_clean":"2023-11-24T06:27:43.657894+0000","last_became_active":"2023-11-24T06:27:41.385754+0000","last_became_peered":"2023-11-24T06:27:41.385754+0000","last_unstale":"2023-11-24T06:27:43.657894+0000","last_undegraded":"2023-11-24T06:27:43.657894+0000","last_fullsized":"2023-11-24T06:27:43.657894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357781+0000","last_change":"2023-11-24T06:27:42.382235+0000","last_active":"2023-11-24T06:27:43.357781+0000","last_peered":"2023-11-24T06:27:43.357781+0000","last_clean":"2023-11-24T06:27:43.357781+0000","last_became_active":"2023-11-24T06:27:42.380559+0000","last_became_peered":"2023-11-24T06:27:42.380559+0000","last_unstale":"2023-11-24T06:27:43.357781+0000","last_undegraded":"2023-11-24T06:27:43.357781+0000","last_fullsized":"2023-11-24T06:27:43.357781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658434+0000","last_change":"2023-11-24T06:27:41.385695+0000","last_active":"2023-11-24T06:27:43.658434+0000","last_peered":"2023-11-24T06:27:43.658434+0000","last_clean":"2023-11-24T06:27:43.658434+0000","last_became_active":"2023-11-24T06:27:41.385158+0000","last_became_peered":"2023-11-24T06:27:41.385158+0000","last_unstale":"2023-11-24T06:27:43.658434+0000","last_undegraded":"2023-11-24T06:27:43.658434+0000","last_fullsized":"2023-11-24T06:27:43.658434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660564+0000","last_change":"2023-11-24T06:27:42.396117+0000","last_active":"2023-11-24T06:27:43.660564+0000","last_peered":"2023-11-24T06:27:43.660564+0000","last_clean":"2023-11-24T06:27:43.660564+0000","last_became_active":"2023-11-24T06:27:42.395511+0000","last_became_peered":"2023-11-24T06:27:42.395511+0000","last_unstale":"2023-11-24T06:27:43.660564+0000","last_undegraded":"2023-11-24T06:27:43.660564+0000","last_fullsized":"2023-11-24T06:27:43.660564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356285+0000","last_change":"2023-11-24T06:27:41.373126+0000","last_active":"2023-11-24T06:27:43.356285+0000","last_peered":"2023-11-24T06:27:43.356285+0000","last_clean":"2023-11-24T06:27:43.356285+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356285+0000","last_undegraded":"2023-11-24T06:27:43.356285+0000","last_fullsized":"2023-11-24T06:27:43.356285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:05.485619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356095+0000","last_change":"2023-11-24T06:27:42.386786+0000","last_active":"2023-11-24T06:27:43.356095+0000","last_peered":"2023-11-24T06:27:43.356095+0000","last_clean":"2023-11-24T06:27:43.356095+0000","last_became_active":"2023-11-24T06:27:42.386492+0000","last_became_peered":"2023-11-24T06:27:42.386492+0000","last_unstale":"2023-11-24T06:27:43.356095+0000","last_undegraded":"2023-11-24T06:27:43.356095+0000","last_fullsized":"2023-11-24T06:27:43.356095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:36:11.379388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.661047+0000","last_change":"2023-11-24T06:27:41.378175+0000","last_active":"2023-11-24T06:27:43.661047+0000","last_peered":"2023-11-24T06:27:43.661047+0000","last_clean":"2023-11-24T06:27:43.661047+0000","last_became_active":"2023-11-24T06:27:41.376774+0000","last_became_peered":"2023-11-24T06:27:41.376774+0000","last_unstale":"2023-11-24T06:27:43.661047+0000","last_undegraded":"2023-11-24T06:27:43.661047+0000","last_fullsized":"2023-11-24T06:27:43.661047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":823296,"data_stored":669752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.282}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.607}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.853}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":72662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2440000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.508}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2949999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:47.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.507+0000 7fc0e5851640 1 Processor -- start 2023-11-24T06:27:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.508+0000 7fc0e5851640 1 -- start start 2023-11-24T06:27:47.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0e5851640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e0160f80 0x7fc0e0161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0e5851640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0e00ce8f0 0x7fc0e00ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0e5851640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e01618a0 con 0x7fc0e00cfbe0 2023-11-24T06:27:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0e5851640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e00d0de0 con 0x7fc0e00ce8f0 2023-11-24T06:27:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0e5851640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00d10c0 con 0x7fc0e0160f80 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0d7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0e00ce8f0 0x7fc0e00ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0df7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc0e00cfbe0 0x7fc0e00d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44726/0 (socket says 172.21.15.64:44726) 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.509+0000 7fc0df7fe640 1 -- 172.21.15.64:0/375573142 learned_addr learned my addr 172.21.15.64:0/375573142 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.510+0000 7fc0deffd640 1 --2- 172.21.15.64:0/375573142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e0160f80 0x7fc0e0161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.510+0000 7fc0de7fc640 1 -- 172.21.15.64:0/375573142 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764562924 0 0) 0x7fc0e01618a0 con 0x7fc0e00cfbe0 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.510+0000 7fc0de7fc640 1 -- 172.21.15.64:0/375573142 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0cc003520 con 0x7fc0e00cfbe0 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.510+0000 7fc0d7fff640 1 -- 172.21.15.64:0/375573142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e0160f80 msgr2=0x7fc0e0161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.510+0000 7fc0d7fff640 1 --2- 172.21.15.64:0/375573142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e0160f80 0x7fc0e0161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.510+0000 7fc0d7fff640 1 -- 172.21.15.64:0/375573142 >> v1:172.21.15.64:6789/0 conn(0x7fc0e00cfbe0 legacy=0x7fc0e00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.510+0000 7fc0d7fff640 1 -- 172.21.15.64:0/375573142 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00d13a0 con 0x7fc0e00ce8f0 2023-11-24T06:27:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.512+0000 7fc0d7fff640 1 --2- 172.21.15.64:0/375573142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e00ce8f0 0x7fc0e00ced50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80097a0 tx=0x7fc0c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac2a0fcd4a77104 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.512+0000 7fc0de7fc640 1 -- 172.21.15.64:0/375573142 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8031830 con 0x7fc0e00ce8f0 2023-11-24T06:27:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.512+0000 7fc0de7fc640 1 -- 172.21.15.64:0/375573142 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c8031990 con 0x7fc0e00ce8f0 2023-11-24T06:27:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.512+0000 7fc0de7fc640 1 -- 172.21.15.64:0/375573142 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8031d20 con 0x7fc0e00ce8f0 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.512+0000 7fc0e5851640 1 -- 172.21.15.64:0/375573142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e00ce8f0 msgr2=0x7fc0e00ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.512+0000 7fc0e5851640 1 --2- 172.21.15.64:0/375573142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e00ce8f0 0x7fc0e00ced50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80097a0 tx=0x7fc0c802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.513+0000 7fc0e5851640 1 -- 172.21.15.64:0/375573142 shutdown_connections 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.513+0000 7fc0e5851640 1 --2- 172.21.15.64:0/375573142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e00ce8f0 0x7fc0e00ced50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.513+0000 7fc0e5851640 1 --2- 172.21.15.64:0/375573142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e0160f80 0x7fc0e0161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.513+0000 7fc0e5851640 1 -- 172.21.15.64:0/375573142 >> 172.21.15.64:0/375573142 conn(0x7fc0e009c550 msgr2=0x7fc0e00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.513+0000 7fc0e5851640 1 -- 172.21.15.64:0/375573142 shutdown_connections 2023-11-24T06:27:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.513+0000 7fc0e5851640 1 -- 172.21.15.64:0/375573142 wait complete. 2023-11-24T06:27:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.515+0000 7fc0e5851640 1 Processor -- start 2023-11-24T06:27:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.515+0000 7fc0e5851640 1 -- start start 2023-11-24T06:27:47.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0e5851640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00ce8f0 0x7fc0e014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0e5851640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cfbe0 0x7fc0e01613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0e5851640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e0160f80 0x7fc0e0161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0e5851640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015fa20 con 0x7fc0e0160f80 2023-11-24T06:27:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0e5851640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015fd00 con 0x7fc0e00ce8f0 2023-11-24T06:27:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0e5851640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015ffe0 con 0x7fc0e00cfbe0 2023-11-24T06:27:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e0160f80 0x7fc0e0161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e0160f80 0x7fc0e0161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48174/0 (socket says 172.21.15.64:48174) 2023-11-24T06:27:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0df7fe640 1 -- 172.21.15.64:0/982474468 learned_addr learned my addr 172.21.15.64:0/982474468 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0d7fff640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cfbe0 0x7fc0e01613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0deffd640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00ce8f0 0x7fc0e014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0df7fe640 1 -- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cfbe0 msgr2=0x7fc0e01613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0df7fe640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cfbe0 0x7fc0e01613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0df7fe640 1 -- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00ce8f0 msgr2=0x7fc0e014e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.516+0000 7fc0df7fe640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00ce8f0 0x7fc0e014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.517+0000 7fc0df7fe640 1 -- 172.21.15.64:0/982474468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c8009400 con 0x7fc0e0160f80 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.517+0000 7fc0df7fe640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e0160f80 0x7fc0e0161910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800cb40 tx=0x7fc0d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.517+0000 7fc0dcff9640 1 -- 172.21.15.64:0/982474468 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800aca0 con 0x7fc0e0160f80 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.517+0000 7fc0dcff9640 1 -- 172.21.15.64:0/982474468 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d800ae00 con 0x7fc0e0160f80 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.517+0000 7fc0dcff9640 1 -- 172.21.15.64:0/982474468 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8012620 con 0x7fc0e0160f80 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.517+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e01602c0 con 0x7fc0e0160f80 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.517+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e015eef0 con 0x7fc0e0160f80 2023-11-24T06:27:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.519+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e00675d0 con 0x7fc0e0160f80 2023-11-24T06:27:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.519+0000 7fc0dcff9640 1 -- 172.21.15.64:0/982474468 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0d8007070 con 0x7fc0e0160f80 2023-11-24T06:27:47.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.520+0000 7fc0dcff9640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc0c006ae40 0x7fc0c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.520+0000 7fc0dcff9640 1 -- 172.21.15.64:0/982474468 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0d8089640 con 0x7fc0e0160f80 2023-11-24T06:27:47.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.521+0000 7fc0deffd640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc0c006ae40 0x7fc0c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.521+0000 7fc0deffd640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc0c006ae40 0x7fc0c006d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009a10 tx=0x7fc0d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.525+0000 7fc0dcff9640 1 -- 172.21.15.64:0/982474468 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0d8054700 con 0x7fc0e0160f80 2023-11-24T06:27:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.716+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0e015fe70 con 0x7fc0c006ae40 2023-11-24T06:27:47.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.727+0000 7fc0dcff9640 1 -- 172.21.15.64:0/982474468 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264500 (secure 0 0 0) 0x7fc0e015fe70 con 0x7fc0c006ae40 2023-11-24T06:27:47.735 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:27:47.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc0c006ae40 msgr2=0x7fc0c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc0c006ae40 0x7fc0c006d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009a10 tx=0x7fc0d0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e0160f80 msgr2=0x7fc0e0161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e0160f80 0x7fc0e0161910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800cb40 tx=0x7fc0d800a060 comp rx=0 tx=0).stop 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 shutdown_connections 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc0c006ae40 0x7fc0c006d300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0e0160f80 0x7fc0e0161910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cfbe0 0x7fc0e01613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 --2- 172.21.15.64:0/982474468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00ce8f0 0x7fc0e014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.733+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 >> 172.21.15.64:0/982474468 conn(0x7fc0e009c550 msgr2=0x7fc0e00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.734+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 shutdown_connections 2023-11-24T06:27:47.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:47.734+0000 7fc0e5851640 1 -- 172.21.15.64:0/982474468 wait complete. 2023-11-24T06:27:47.759 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T06:27:47.102569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219312,"kb_used_data":2972,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755408,"statfs":{"total":773094113280,"available":772869537792,"internally_reserved":0,"allocated":3043328,"data_stored":1809672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002124"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658921+0000","last_change":"2023-11-24T06:27:42.394622+0000","last_active":"2023-11-24T06:27:43.658921+0000","last_peered":"2023-11-24T06:27:43.658921+0000","last_clean":"2023-11-24T06:27:43.658921+0000","last_became_active":"2023-11-24T06:27:42.394278+0000","last_became_peered":"2023-11-24T06:27:42.394278+0000","last_unstale":"2023-11-24T06:27:43.658921+0000","last_undegraded":"2023-11-24T06:27:43.658921+0000","last_fullsized":"2023-11-24T06:27:43.658921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659042+0000","last_change":"2023-11-24T06:27:42.395699+0000","last_active":"2023-11-24T06:27:43.659042+0000","last_peered":"2023-11-24T06:27:43.659042+0000","last_clean":"2023-11-24T06:27:43.659042+0000","last_became_active":"2023-11-24T06:27:42.395424+0000","last_became_peered":"2023-11-24T06:27:42.395424+0000","last_unstale":"2023-11-24T06:27:43.659042+0000","last_undegraded":"2023-11-24T06:27:43.659042+0000","last_fullsized":"2023-11-24T06:27:43.659042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660634+0000","last_change":"2023-11-24T06:27:42.390431+0000","last_active":"2023-11-24T06:27:43.660634+0000","last_peered":"2023-11-24T06:27:43.660634+0000","last_clean":"2023-11-24T06:27:43.660634+0000","last_became_active":"2023-11-24T06:27:42.390163+0000","last_became_peered":"2023-11-24T06:27:42.390163+0000","last_unstale":"2023-11-24T06:27:43.660634+0000","last_undegraded":"2023-11-24T06:27:43.660634+0000","last_fullsized":"2023-11-24T06:27:43.660634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357785+0000","last_change":"2023-11-24T06:27:42.393940+0000","last_active":"2023-11-24T06:27:43.357785+0000","last_peered":"2023-11-24T06:27:43.357785+0000","last_clean":"2023-11-24T06:27:43.357785+0000","last_became_active":"2023-11-24T06:27:42.393341+0000","last_became_peered":"2023-11-24T06:27:42.393341+0000","last_unstale":"2023-11-24T06:27:43.357785+0000","last_undegraded":"2023-11-24T06:27:43.357785+0000","last_fullsized":"2023-11-24T06:27:43.357785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658343+0000","last_change":"2023-11-24T06:27:42.392760+0000","last_active":"2023-11-24T06:27:43.658343+0000","last_peered":"2023-11-24T06:27:43.658343+0000","last_clean":"2023-11-24T06:27:43.658343+0000","last_became_active":"2023-11-24T06:27:42.390011+0000","last_became_peered":"2023-11-24T06:27:42.390011+0000","last_unstale":"2023-11-24T06:27:43.658343+0000","last_undegraded":"2023-11-24T06:27:43.658343+0000","last_fullsized":"2023-11-24T06:27:43.658343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355889+0000","last_change":"2023-11-24T06:27:42.397895+0000","last_active":"2023-11-24T06:27:43.355889+0000","last_peered":"2023-11-24T06:27:43.355889+0000","last_clean":"2023-11-24T06:27:43.355889+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355889+0000","last_undegraded":"2023-11-24T06:27:43.355889+0000","last_fullsized":"2023-11-24T06:27:43.355889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657379+0000","last_change":"2023-11-24T06:27:42.398329+0000","last_active":"2023-11-24T06:27:43.657379+0000","last_peered":"2023-11-24T06:27:43.657379+0000","last_clean":"2023-11-24T06:27:43.657379+0000","last_became_active":"2023-11-24T06:27:42.398095+0000","last_became_peered":"2023-11-24T06:27:42.398095+0000","last_unstale":"2023-11-24T06:27:43.657379+0000","last_undegraded":"2023-11-24T06:27:43.657379+0000","last_fullsized":"2023-11-24T06:27:43.657379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355033+0000","last_change":"2023-11-24T06:27:42.395940+0000","last_active":"2023-11-24T06:27:43.355033+0000","last_peered":"2023-11-24T06:27:43.355033+0000","last_clean":"2023-11-24T06:27:43.355033+0000","last_became_active":"2023-11-24T06:27:42.395595+0000","last_became_peered":"2023-11-24T06:27:42.395595+0000","last_unstale":"2023-11-24T06:27:43.355033+0000","last_undegraded":"2023-11-24T06:27:43.355033+0000","last_fullsized":"2023-11-24T06:27:43.355033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659266+0000","last_change":"2023-11-24T06:27:42.394418+0000","last_active":"2023-11-24T06:27:43.659266+0000","last_peered":"2023-11-24T06:27:43.659266+0000","last_clean":"2023-11-24T06:27:43.659266+0000","last_became_active":"2023-11-24T06:27:42.394103+0000","last_became_peered":"2023-11-24T06:27:42.394103+0000","last_unstale":"2023-11-24T06:27:43.659266+0000","last_undegraded":"2023-11-24T06:27:43.659266+0000","last_fullsized":"2023-11-24T06:27:43.659266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658516+0000","last_change":"2023-11-24T06:27:42.386091+0000","last_active":"2023-11-24T06:27:43.658516+0000","last_peered":"2023-11-24T06:27:43.658516+0000","last_clean":"2023-11-24T06:27:43.658516+0000","last_became_active":"2023-11-24T06:27:42.385638+0000","last_became_peered":"2023-11-24T06:27:42.385638+0000","last_unstale":"2023-11-24T06:27:43.658516+0000","last_undegraded":"2023-11-24T06:27:43.658516+0000","last_fullsized":"2023-11-24T06:27:43.658516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657666+0000","last_change":"2023-11-24T06:27:42.385847+0000","last_active":"2023-11-24T06:27:43.657666+0000","last_peered":"2023-11-24T06:27:43.657666+0000","last_clean":"2023-11-24T06:27:43.657666+0000","last_became_active":"2023-11-24T06:27:42.384548+0000","last_became_peered":"2023-11-24T06:27:42.384548+0000","last_unstale":"2023-11-24T06:27:43.657666+0000","last_undegraded":"2023-11-24T06:27:43.657666+0000","last_fullsized":"2023-11-24T06:27:43.657666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359608+0000","last_change":"2023-11-24T06:27:42.381135+0000","last_active":"2023-11-24T06:27:43.359608+0000","last_peered":"2023-11-24T06:27:43.359608+0000","last_clean":"2023-11-24T06:27:43.359608+0000","last_became_active":"2023-11-24T06:27:42.379951+0000","last_became_peered":"2023-11-24T06:27:42.379951+0000","last_unstale":"2023-11-24T06:27:43.359608+0000","last_undegraded":"2023-11-24T06:27:43.359608+0000","last_fullsized":"2023-11-24T06:27:43.359608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355537+0000","last_change":"2023-11-24T06:27:42.396365+0000","last_active":"2023-11-24T06:27:43.355537+0000","last_peered":"2023-11-24T06:27:43.355537+0000","last_clean":"2023-11-24T06:27:43.355537+0000","last_became_active":"2023-11-24T06:27:42.394255+0000","last_became_peered":"2023-11-24T06:27:42.394255+0000","last_unstale":"2023-11-24T06:27:43.355537+0000","last_undegraded":"2023-11-24T06:27:43.355537+0000","last_fullsized":"2023-11-24T06:27:43.355537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660061+0000","last_change":"2023-11-24T06:27:42.385401+0000","last_active":"2023-11-24T06:27:43.660061+0000","last_peered":"2023-11-24T06:27:43.660061+0000","last_clean":"2023-11-24T06:27:43.660061+0000","last_became_active":"2023-11-24T06:27:42.384001+0000","last_became_peered":"2023-11-24T06:27:42.384001+0000","last_unstale":"2023-11-24T06:27:43.660061+0000","last_undegraded":"2023-11-24T06:27:43.660061+0000","last_fullsized":"2023-11-24T06:27:43.660061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:50.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657883+0000","last_change":"2023-11-24T06:27:42.375262+0000","last_active":"2023-11-24T06:27:43.657883+0000","last_peered":"2023-11-24T06:27:43.657883+0000","last_clean":"2023-11-24T06:27:43.657883+0000","last_became_active":"2023-11-24T06:27:42.374748+0000","last_became_peered":"2023-11-24T06:27:42.374748+0000","last_unstale":"2023-11-24T06:27:43.657883+0000","last_undegraded":"2023-11-24T06:27:43.657883+0000","last_fullsized":"2023-11-24T06:27:43.657883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355370+0000","last_change":"2023-11-24T06:27:42.397612+0000","last_active":"2023-11-24T06:27:43.355370+0000","last_peered":"2023-11-24T06:27:43.355370+0000","last_clean":"2023-11-24T06:27:43.355370+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355370+0000","last_undegraded":"2023-11-24T06:27:43.355370+0000","last_fullsized":"2023-11-24T06:27:43.355370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359434+0000","last_change":"2023-11-24T06:27:42.388744+0000","last_active":"2023-11-24T06:27:43.359434+0000","last_peered":"2023-11-24T06:27:43.359434+0000","last_clean":"2023-11-24T06:27:43.359434+0000","last_became_active":"2023-11-24T06:27:42.388298+0000","last_became_peered":"2023-11-24T06:27:42.388298+0000","last_unstale":"2023-11-24T06:27:43.359434+0000","last_undegraded":"2023-11-24T06:27:43.359434+0000","last_fullsized":"2023-11-24T06:27:43.359434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658048+0000","last_change":"2023-11-24T06:27:42.377765+0000","last_active":"2023-11-24T06:27:43.658048+0000","last_peered":"2023-11-24T06:27:43.658048+0000","last_clean":"2023-11-24T06:27:43.658048+0000","last_became_active":"2023-11-24T06:27:42.377032+0000","last_became_peered":"2023-11-24T06:27:42.377032+0000","last_unstale":"2023-11-24T06:27:43.658048+0000","last_undegraded":"2023-11-24T06:27:43.658048+0000","last_fullsized":"2023-11-24T06:27:43.658048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658694+0000","last_change":"2023-11-24T06:27:42.390624+0000","last_active":"2023-11-24T06:27:43.658694+0000","last_peered":"2023-11-24T06:27:43.658694+0000","last_clean":"2023-11-24T06:27:43.658694+0000","last_became_active":"2023-11-24T06:27:42.387611+0000","last_became_peered":"2023-11-24T06:27:42.387611+0000","last_unstale":"2023-11-24T06:27:43.658694+0000","last_undegraded":"2023-11-24T06:27:43.658694+0000","last_fullsized":"2023-11-24T06:27:43.658694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658079+0000","last_change":"2023-11-24T06:27:42.384846+0000","last_active":"2023-11-24T06:27:43.658079+0000","last_peered":"2023-11-24T06:27:43.658079+0000","last_clean":"2023-11-24T06:27:43.658079+0000","last_became_active":"2023-11-24T06:27:42.384575+0000","last_became_peered":"2023-11-24T06:27:42.384575+0000","last_unstale":"2023-11-24T06:27:43.658079+0000","last_undegraded":"2023-11-24T06:27:43.658079+0000","last_fullsized":"2023-11-24T06:27:43.658079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660460+0000","last_change":"2023-11-24T06:27:42.393160+0000","last_active":"2023-11-24T06:27:43.660460+0000","last_peered":"2023-11-24T06:27:43.660460+0000","last_clean":"2023-11-24T06:27:43.660460+0000","last_became_active":"2023-11-24T06:27:42.392277+0000","last_became_peered":"2023-11-24T06:27:42.392277+0000","last_unstale":"2023-11-24T06:27:43.660460+0000","last_undegraded":"2023-11-24T06:27:43.660460+0000","last_fullsized":"2023-11-24T06:27:43.660460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355922+0000","last_change":"2023-11-24T06:27:42.390004+0000","last_active":"2023-11-24T06:27:43.355922+0000","last_peered":"2023-11-24T06:27:43.355922+0000","last_clean":"2023-11-24T06:27:43.355922+0000","last_became_active":"2023-11-24T06:27:42.389682+0000","last_became_peered":"2023-11-24T06:27:42.389682+0000","last_unstale":"2023-11-24T06:27:43.355922+0000","last_undegraded":"2023-11-24T06:27:43.355922+0000","last_fullsized":"2023-11-24T06:27:43.355922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355110+0000","last_change":"2023-11-24T06:27:42.380022+0000","last_active":"2023-11-24T06:27:43.355110+0000","last_peered":"2023-11-24T06:27:43.355110+0000","last_clean":"2023-11-24T06:27:43.355110+0000","last_became_active":"2023-11-24T06:27:42.379510+0000","last_became_peered":"2023-11-24T06:27:42.379510+0000","last_unstale":"2023-11-24T06:27:43.355110+0000","last_undegraded":"2023-11-24T06:27:43.355110+0000","last_fullsized":"2023-11-24T06:27:43.355110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355598+0000","last_change":"2023-11-24T06:27:42.392966+0000","last_active":"2023-11-24T06:27:43.355598+0000","last_peered":"2023-11-24T06:27:43.355598+0000","last_clean":"2023-11-24T06:27:43.355598+0000","last_became_active":"2023-11-24T06:27:42.392683+0000","last_became_peered":"2023-11-24T06:27:42.392683+0000","last_unstale":"2023-11-24T06:27:43.355598+0000","last_undegraded":"2023-11-24T06:27:43.355598+0000","last_fullsized":"2023-11-24T06:27:43.355598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356153+0000","last_change":"2023-11-24T06:27:42.384496+0000","last_active":"2023-11-24T06:27:43.356153+0000","last_peered":"2023-11-24T06:27:43.356153+0000","last_clean":"2023-11-24T06:27:43.356153+0000","last_became_active":"2023-11-24T06:27:42.384121+0000","last_became_peered":"2023-11-24T06:27:42.384121+0000","last_unstale":"2023-11-24T06:27:43.356153+0000","last_undegraded":"2023-11-24T06:27:43.356153+0000","last_fullsized":"2023-11-24T06:27:43.356153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:09.683680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658776+0000","last_change":"2023-11-24T06:27:42.386718+0000","last_active":"2023-11-24T06:27:43.658776+0000","last_peered":"2023-11-24T06:27:43.658776+0000","last_clean":"2023-11-24T06:27:43.658776+0000","last_became_active":"2023-11-24T06:27:42.386397+0000","last_became_peered":"2023-11-24T06:27:42.386397+0000","last_unstale":"2023-11-24T06:27:43.658776+0000","last_undegraded":"2023-11-24T06:27:43.658776+0000","last_fullsized":"2023-11-24T06:27:43.658776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658422+0000","last_change":"2023-11-24T06:27:42.389793+0000","last_active":"2023-11-24T06:27:43.658422+0000","last_peered":"2023-11-24T06:27:43.658422+0000","last_clean":"2023-11-24T06:27:43.658422+0000","last_became_active":"2023-11-24T06:27:42.389493+0000","last_became_peered":"2023-11-24T06:27:42.389493+0000","last_unstale":"2023-11-24T06:27:43.658422+0000","last_undegraded":"2023-11-24T06:27:43.658422+0000","last_fullsized":"2023-11-24T06:27:43.658422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356062+0000","last_change":"2023-11-24T06:27:42.389497+0000","last_active":"2023-11-24T06:27:43.356062+0000","last_peered":"2023-11-24T06:27:43.356062+0000","last_clean":"2023-11-24T06:27:43.356062+0000","last_became_active":"2023-11-24T06:27:42.389132+0000","last_became_peered":"2023-11-24T06:27:42.389132+0000","last_unstale":"2023-11-24T06:27:43.356062+0000","last_undegraded":"2023-11-24T06:27:43.356062+0000","last_fullsized":"2023-11-24T06:27:43.356062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:37.697060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660457+0000","last_change":"2023-11-24T06:27:42.398466+0000","last_active":"2023-11-24T06:27:43.660457+0000","last_peered":"2023-11-24T06:27:43.660457+0000","last_clean":"2023-11-24T06:27:43.660457+0000","last_became_active":"2023-11-24T06:27:42.397061+0000","last_became_peered":"2023-11-24T06:27:42.397061+0000","last_unstale":"2023-11-24T06:27:43.660457+0000","last_undegraded":"2023-11-24T06:27:43.660457+0000","last_fullsized":"2023-11-24T06:27:43.660457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356264+0000","last_change":"2023-11-24T06:27:42.385278+0000","last_active":"2023-11-24T06:27:43.356264+0000","last_peered":"2023-11-24T06:27:43.356264+0000","last_clean":"2023-11-24T06:27:43.356264+0000","last_became_active":"2023-11-24T06:27:42.384909+0000","last_became_peered":"2023-11-24T06:27:42.384909+0000","last_unstale":"2023-11-24T06:27:43.356264+0000","last_undegraded":"2023-11-24T06:27:43.356264+0000","last_fullsized":"2023-11-24T06:27:43.356264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656675+0000","last_change":"2023-11-24T06:27:42.382417+0000","last_active":"2023-11-24T06:27:43.656675+0000","last_peered":"2023-11-24T06:27:43.656675+0000","last_clean":"2023-11-24T06:27:43.656675+0000","last_became_active":"2023-11-24T06:27:42.381985+0000","last_became_peered":"2023-11-24T06:27:42.381985+0000","last_unstale":"2023-11-24T06:27:43.656675+0000","last_undegraded":"2023-11-24T06:27:43.656675+0000","last_fullsized":"2023-11-24T06:27:43.656675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656964+0000","last_change":"2023-11-24T06:27:42.395403+0000","last_active":"2023-11-24T06:27:43.656964+0000","last_peered":"2023-11-24T06:27:43.656964+0000","last_clean":"2023-11-24T06:27:43.656964+0000","last_became_active":"2023-11-24T06:27:42.394442+0000","last_became_peered":"2023-11-24T06:27:42.394442+0000","last_unstale":"2023-11-24T06:27:43.656964+0000","last_undegraded":"2023-11-24T06:27:43.656964+0000","last_fullsized":"2023-11-24T06:27:43.656964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357092+0000","last_change":"2023-11-24T06:27:42.375948+0000","last_active":"2023-11-24T06:27:43.357092+0000","last_peered":"2023-11-24T06:27:43.357092+0000","last_clean":"2023-11-24T06:27:43.357092+0000","last_became_active":"2023-11-24T06:27:42.374443+0000","last_became_peered":"2023-11-24T06:27:42.374443+0000","last_unstale":"2023-11-24T06:27:43.357092+0000","last_undegraded":"2023-11-24T06:27:43.357092+0000","last_fullsized":"2023-11-24T06:27:43.357092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:11.141034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355607+0000","last_change":"2023-11-24T06:27:41.381254+0000","last_active":"2023-11-24T06:27:43.355607+0000","last_peered":"2023-11-24T06:27:43.355607+0000","last_clean":"2023-11-24T06:27:43.355607+0000","last_became_active":"2023-11-24T06:27:41.380768+0000","last_became_peered":"2023-11-24T06:27:41.380768+0000","last_unstale":"2023-11-24T06:27:43.355607+0000","last_undegraded":"2023-11-24T06:27:43.355607+0000","last_fullsized":"2023-11-24T06:27:43.355607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660887+0000","last_change":"2023-11-24T06:27:42.393018+0000","last_active":"2023-11-24T06:27:43.660887+0000","last_peered":"2023-11-24T06:27:43.660887+0000","last_clean":"2023-11-24T06:27:43.660887+0000","last_became_active":"2023-11-24T06:27:42.391726+0000","last_became_peered":"2023-11-24T06:27:42.391726+0000","last_unstale":"2023-11-24T06:27:43.660887+0000","last_undegraded":"2023-11-24T06:27:43.660887+0000","last_fullsized":"2023-11-24T06:27:43.660887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657580+0000","last_change":"2023-11-24T06:27:41.383020+0000","last_active":"2023-11-24T06:27:43.657580+0000","last_peered":"2023-11-24T06:27:43.657580+0000","last_clean":"2023-11-24T06:27:43.657580+0000","last_became_active":"2023-11-24T06:27:41.382642+0000","last_became_peered":"2023-11-24T06:27:41.382642+0000","last_unstale":"2023-11-24T06:27:43.657580+0000","last_undegraded":"2023-11-24T06:27:43.657580+0000","last_fullsized":"2023-11-24T06:27:43.657580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:19.408882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355377+0000","last_change":"2023-11-24T06:27:41.382979+0000","last_active":"2023-11-24T06:27:43.355377+0000","last_peered":"2023-11-24T06:27:43.355377+0000","last_clean":"2023-11-24T06:27:43.355377+0000","last_became_active":"2023-11-24T06:27:41.381955+0000","last_became_peered":"2023-11-24T06:27:41.381955+0000","last_unstale":"2023-11-24T06:27:43.355377+0000","last_undegraded":"2023-11-24T06:27:43.355377+0000","last_fullsized":"2023-11-24T06:27:43.355377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.354996+0000","last_change":"2023-11-24T06:27:42.395234+0000","last_active":"2023-11-24T06:27:43.354996+0000","last_peered":"2023-11-24T06:27:43.354996+0000","last_clean":"2023-11-24T06:27:43.354996+0000","last_became_active":"2023-11-24T06:27:42.394890+0000","last_became_peered":"2023-11-24T06:27:42.394890+0000","last_unstale":"2023-11-24T06:27:43.354996+0000","last_undegraded":"2023-11-24T06:27:43.354996+0000","last_fullsized":"2023-11-24T06:27:43.354996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355626+0000","last_change":"2023-11-24T06:27:41.371070+0000","last_active":"2023-11-24T06:27:43.355626+0000","last_peered":"2023-11-24T06:27:43.355626+0000","last_clean":"2023-11-24T06:27:43.355626+0000","last_became_active":"2023-11-24T06:27:41.368037+0000","last_became_peered":"2023-11-24T06:27:41.368037+0000","last_unstale":"2023-11-24T06:27:43.355626+0000","last_undegraded":"2023-11-24T06:27:43.355626+0000","last_fullsized":"2023-11-24T06:27:43.355626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657673+0000","last_change":"2023-11-24T06:27:42.385239+0000","last_active":"2023-11-24T06:27:43.657673+0000","last_peered":"2023-11-24T06:27:43.657673+0000","last_clean":"2023-11-24T06:27:43.657673+0000","last_became_active":"2023-11-24T06:27:42.384436+0000","last_became_peered":"2023-11-24T06:27:42.384436+0000","last_unstale":"2023-11-24T06:27:43.657673+0000","last_undegraded":"2023-11-24T06:27:43.657673+0000","last_fullsized":"2023-11-24T06:27:43.657673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657925+0000","last_change":"2023-11-24T06:27:41.381169+0000","last_active":"2023-11-24T06:27:43.657925+0000","last_peered":"2023-11-24T06:27:43.657925+0000","last_clean":"2023-11-24T06:27:43.657925+0000","last_became_active":"2023-11-24T06:27:41.380835+0000","last_became_peered":"2023-11-24T06:27:41.380835+0000","last_unstale":"2023-11-24T06:27:43.657925+0000","last_undegraded":"2023-11-24T06:27:43.657925+0000","last_fullsized":"2023-11-24T06:27:43.657925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358731+0000","last_change":"2023-11-24T06:27:42.390319+0000","last_active":"2023-11-24T06:27:43.358731+0000","last_peered":"2023-11-24T06:27:43.358731+0000","last_clean":"2023-11-24T06:27:43.358731+0000","last_became_active":"2023-11-24T06:27:42.388441+0000","last_became_peered":"2023-11-24T06:27:42.388441+0000","last_unstale":"2023-11-24T06:27:43.358731+0000","last_undegraded":"2023-11-24T06:27:43.358731+0000","last_fullsized":"2023-11-24T06:27:43.358731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358844+0000","last_change":"2023-11-24T06:27:41.385916+0000","last_active":"2023-11-24T06:27:43.358844+0000","last_peered":"2023-11-24T06:27:43.358844+0000","last_clean":"2023-11-24T06:27:43.358844+0000","last_became_active":"2023-11-24T06:27:41.385609+0000","last_became_peered":"2023-11-24T06:27:41.385609+0000","last_unstale":"2023-11-24T06:27:43.358844+0000","last_undegraded":"2023-11-24T06:27:43.358844+0000","last_fullsized":"2023-11-24T06:27:43.358844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660892+0000","last_change":"2023-11-24T06:27:42.395957+0000","last_active":"2023-11-24T06:27:43.660892+0000","last_peered":"2023-11-24T06:27:43.660892+0000","last_clean":"2023-11-24T06:27:43.660892+0000","last_became_active":"2023-11-24T06:27:42.395222+0000","last_became_peered":"2023-11-24T06:27:42.395222+0000","last_unstale":"2023-11-24T06:27:43.660892+0000","last_undegraded":"2023-11-24T06:27:43.660892+0000","last_fullsized":"2023-11-24T06:27:43.660892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658670+0000","last_change":"2023-11-24T06:27:41.382629+0000","last_active":"2023-11-24T06:27:43.658670+0000","last_peered":"2023-11-24T06:27:43.658670+0000","last_clean":"2023-11-24T06:27:43.658670+0000","last_became_active":"2023-11-24T06:27:41.382323+0000","last_became_peered":"2023-11-24T06:27:41.382323+0000","last_unstale":"2023-11-24T06:27:43.658670+0000","last_undegraded":"2023-11-24T06:27:43.658670+0000","last_fullsized":"2023-11-24T06:27:43.658670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:38.362359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359838+0000","last_change":"2023-11-24T06:27:42.373431+0000","last_active":"2023-11-24T06:27:43.359838+0000","last_peered":"2023-11-24T06:27:43.359838+0000","last_clean":"2023-11-24T06:27:43.359838+0000","last_became_active":"2023-11-24T06:27:42.373106+0000","last_became_peered":"2023-11-24T06:27:42.373106+0000","last_unstale":"2023-11-24T06:27:43.359838+0000","last_undegraded":"2023-11-24T06:27:43.359838+0000","last_fullsized":"2023-11-24T06:27:43.359838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658482+0000","last_change":"2023-11-24T06:27:41.381556+0000","last_active":"2023-11-24T06:27:43.658482+0000","last_peered":"2023-11-24T06:27:43.658482+0000","last_clean":"2023-11-24T06:27:43.658482+0000","last_became_active":"2023-11-24T06:27:41.380552+0000","last_became_peered":"2023-11-24T06:27:41.380552+0000","last_unstale":"2023-11-24T06:27:43.658482+0000","last_undegraded":"2023-11-24T06:27:43.658482+0000","last_fullsized":"2023-11-24T06:27:43.658482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:48.968587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659001+0000","last_change":"2023-11-24T06:27:42.391222+0000","last_active":"2023-11-24T06:27:43.659001+0000","last_peered":"2023-11-24T06:27:43.659001+0000","last_clean":"2023-11-24T06:27:43.659001+0000","last_became_active":"2023-11-24T06:27:42.390910+0000","last_became_peered":"2023-11-24T06:27:42.390910+0000","last_unstale":"2023-11-24T06:27:43.659001+0000","last_undegraded":"2023-11-24T06:27:43.659001+0000","last_fullsized":"2023-11-24T06:27:43.659001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660728+0000","last_change":"2023-11-24T06:27:41.374697+0000","last_active":"2023-11-24T06:27:43.660728+0000","last_peered":"2023-11-24T06:27:43.660728+0000","last_clean":"2023-11-24T06:27:43.660728+0000","last_became_active":"2023-11-24T06:27:41.374419+0000","last_became_peered":"2023-11-24T06:27:41.374419+0000","last_unstale":"2023-11-24T06:27:43.660728+0000","last_undegraded":"2023-11-24T06:27:43.660728+0000","last_fullsized":"2023-11-24T06:27:43.660728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660079+0000","last_change":"2023-11-24T06:27:42.393174+0000","last_active":"2023-11-24T06:27:43.660079+0000","last_peered":"2023-11-24T06:27:43.660079+0000","last_clean":"2023-11-24T06:27:43.660079+0000","last_became_active":"2023-11-24T06:27:42.392964+0000","last_became_peered":"2023-11-24T06:27:42.392964+0000","last_unstale":"2023-11-24T06:27:43.660079+0000","last_undegraded":"2023-11-24T06:27:43.660079+0000","last_fullsized":"2023-11-24T06:27:43.660079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356229+0000","last_change":"2023-11-24T06:27:41.379781+0000","last_active":"2023-11-24T06:27:43.356229+0000","last_peered":"2023-11-24T06:27:43.356229+0000","last_clean":"2023-11-24T06:27:43.356229+0000","last_became_active":"2023-11-24T06:27:41.379484+0000","last_became_peered":"2023-11-24T06:27:41.379484+0000","last_unstale":"2023-11-24T06:27:43.356229+0000","last_undegraded":"2023-11-24T06:27:43.356229+0000","last_fullsized":"2023-11-24T06:27:43.356229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658712+0000","last_change":"2023-11-24T06:27:42.385192+0000","last_active":"2023-11-24T06:27:43.658712+0000","last_peered":"2023-11-24T06:27:43.658712+0000","last_clean":"2023-11-24T06:27:43.658712+0000","last_became_active":"2023-11-24T06:27:42.384777+0000","last_became_peered":"2023-11-24T06:27:42.384777+0000","last_unstale":"2023-11-24T06:27:43.658712+0000","last_undegraded":"2023-11-24T06:27:43.658712+0000","last_fullsized":"2023-11-24T06:27:43.658712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657173+0000","last_change":"2023-11-24T06:27:41.383717+0000","last_active":"2023-11-24T06:27:43.657173+0000","last_peered":"2023-11-24T06:27:43.657173+0000","last_clean":"2023-11-24T06:27:43.657173+0000","last_became_active":"2023-11-24T06:27:41.382246+0000","last_became_peered":"2023-11-24T06:27:41.382246+0000","last_unstale":"2023-11-24T06:27:43.657173+0000","last_undegraded":"2023-11-24T06:27:43.657173+0000","last_fullsized":"2023-11-24T06:27:43.657173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:04.357908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656917+0000","last_change":"2023-11-24T06:27:42.392505+0000","last_active":"2023-11-24T06:27:43.656917+0000","last_peered":"2023-11-24T06:27:43.656917+0000","last_clean":"2023-11-24T06:27:43.656917+0000","last_became_active":"2023-11-24T06:27:42.391756+0000","last_became_peered":"2023-11-24T06:27:42.391756+0000","last_unstale":"2023-11-24T06:27:43.656917+0000","last_undegraded":"2023-11-24T06:27:43.656917+0000","last_fullsized":"2023-11-24T06:27:43.656917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656383+0000","last_change":"2023-11-24T06:27:41.366927+0000","last_active":"2023-11-24T06:27:43.656383+0000","last_peered":"2023-11-24T06:27:43.656383+0000","last_clean":"2023-11-24T06:27:43.656383+0000","last_became_active":"2023-11-24T06:27:41.366457+0000","last_became_peered":"2023-11-24T06:27:41.366457+0000","last_unstale":"2023-11-24T06:27:43.656383+0000","last_undegraded":"2023-11-24T06:27:43.656383+0000","last_fullsized":"2023-11-24T06:27:43.656383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657400+0000","last_change":"2023-11-24T06:27:42.386516+0000","last_active":"2023-11-24T06:27:43.657400+0000","last_peered":"2023-11-24T06:27:43.657400+0000","last_clean":"2023-11-24T06:27:43.657400+0000","last_became_active":"2023-11-24T06:27:42.386236+0000","last_became_peered":"2023-11-24T06:27:42.386236+0000","last_unstale":"2023-11-24T06:27:43.657400+0000","last_undegraded":"2023-11-24T06:27:43.657400+0000","last_fullsized":"2023-11-24T06:27:43.657400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358955+0000","last_change":"2023-11-24T06:27:41.368887+0000","last_active":"2023-11-24T06:27:43.358955+0000","last_peered":"2023-11-24T06:27:43.358955+0000","last_clean":"2023-11-24T06:27:43.358955+0000","last_became_active":"2023-11-24T06:27:41.367135+0000","last_became_peered":"2023-11-24T06:27:41.367135+0000","last_unstale":"2023-11-24T06:27:43.358955+0000","last_undegraded":"2023-11-24T06:27:43.358955+0000","last_fullsized":"2023-11-24T06:27:43.358955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:11.584532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659288+0000","last_change":"2023-11-24T06:27:42.391675+0000","last_active":"2023-11-24T06:27:43.659288+0000","last_peered":"2023-11-24T06:27:43.659288+0000","last_clean":"2023-11-24T06:27:43.659288+0000","last_became_active":"2023-11-24T06:27:42.391212+0000","last_became_peered":"2023-11-24T06:27:42.391212+0000","last_unstale":"2023-11-24T06:27:43.659288+0000","last_undegraded":"2023-11-24T06:27:43.659288+0000","last_fullsized":"2023-11-24T06:27:43.659288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658283+0000","last_change":"2023-11-24T06:27:41.377474+0000","last_active":"2023-11-24T06:27:43.658283+0000","last_peered":"2023-11-24T06:27:43.658283+0000","last_clean":"2023-11-24T06:27:43.658283+0000","last_became_active":"2023-11-24T06:27:41.377079+0000","last_became_peered":"2023-11-24T06:27:41.377079+0000","last_unstale":"2023-11-24T06:27:43.658283+0000","last_undegraded":"2023-11-24T06:27:43.658283+0000","last_fullsized":"2023-11-24T06:27:43.658283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659627+0000","last_change":"2023-11-24T06:27:42.388823+0000","last_active":"2023-11-24T06:27:43.659627+0000","last_peered":"2023-11-24T06:27:43.659627+0000","last_clean":"2023-11-24T06:27:43.659627+0000","last_became_active":"2023-11-24T06:27:42.388562+0000","last_became_peered":"2023-11-24T06:27:42.388562+0000","last_unstale":"2023-11-24T06:27:43.659627+0000","last_undegraded":"2023-11-24T06:27:43.659627+0000","last_fullsized":"2023-11-24T06:27:43.659627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:02.353531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658740+0000","last_change":"2023-11-24T06:27:41.375328+0000","last_active":"2023-11-24T06:27:43.658740+0000","last_peered":"2023-11-24T06:27:43.658740+0000","last_clean":"2023-11-24T06:27:43.658740+0000","last_became_active":"2023-11-24T06:27:41.375049+0000","last_became_peered":"2023-11-24T06:27:41.375049+0000","last_unstale":"2023-11-24T06:27:43.658740+0000","last_undegraded":"2023-11-24T06:27:43.658740+0000","last_fullsized":"2023-11-24T06:27:43.658740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358153+0000","last_change":"2023-11-24T06:27:42.398402+0000","last_active":"2023-11-24T06:27:43.358153+0000","last_peered":"2023-11-24T06:27:43.358153+0000","last_clean":"2023-11-24T06:27:43.358153+0000","last_became_active":"2023-11-24T06:27:42.393706+0000","last_became_peered":"2023-11-24T06:27:42.393706+0000","last_unstale":"2023-11-24T06:27:43.358153+0000","last_undegraded":"2023-11-24T06:27:43.358153+0000","last_fullsized":"2023-11-24T06:27:43.358153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657227+0000","last_change":"2023-11-24T06:27:41.366299+0000","last_active":"2023-11-24T06:27:43.657227+0000","last_peered":"2023-11-24T06:27:43.657227+0000","last_clean":"2023-11-24T06:27:43.657227+0000","last_became_active":"2023-11-24T06:27:41.365987+0000","last_became_peered":"2023-11-24T06:27:41.365987+0000","last_unstale":"2023-11-24T06:27:43.657227+0000","last_undegraded":"2023-11-24T06:27:43.657227+0000","last_fullsized":"2023-11-24T06:27:43.657227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:31.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658289+0000","last_change":"2023-11-24T06:27:42.392311+0000","last_active":"2023-11-24T06:27:43.658289+0000","last_peered":"2023-11-24T06:27:43.658289+0000","last_clean":"2023-11-24T06:27:43.658289+0000","last_became_active":"2023-11-24T06:27:42.391517+0000","last_became_peered":"2023-11-24T06:27:42.391517+0000","last_unstale":"2023-11-24T06:27:43.658289+0000","last_undegraded":"2023-11-24T06:27:43.658289+0000","last_fullsized":"2023-11-24T06:27:43.658289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.372912+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:27:42.372912+0000","last_peered":"2023-11-24T06:27:42.372912+0000","last_clean":"2023-11-24T06:27:42.372912+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:27:42.372912+0000","last_undegraded":"2023-11-24T06:27:42.372912+0000","last_fullsized":"2023-11-24T06:27:42.372912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355438+0000","last_change":"2023-11-24T06:27:41.370135+0000","last_active":"2023-11-24T06:27:43.355438+0000","last_peered":"2023-11-24T06:27:43.355438+0000","last_clean":"2023-11-24T06:27:43.355438+0000","last_became_active":"2023-11-24T06:27:41.369372+0000","last_became_peered":"2023-11-24T06:27:41.369372+0000","last_unstale":"2023-11-24T06:27:43.355438+0000","last_undegraded":"2023-11-24T06:27:43.355438+0000","last_fullsized":"2023-11-24T06:27:43.355438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:07.305138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355221+0000","last_change":"2023-11-24T06:27:42.384732+0000","last_active":"2023-11-24T06:27:43.355221+0000","last_peered":"2023-11-24T06:27:43.355221+0000","last_clean":"2023-11-24T06:27:43.355221+0000","last_became_active":"2023-11-24T06:27:42.384083+0000","last_became_peered":"2023-11-24T06:27:42.384083+0000","last_unstale":"2023-11-24T06:27:43.355221+0000","last_undegraded":"2023-11-24T06:27:43.355221+0000","last_fullsized":"2023-11-24T06:27:43.355221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656745+0000","last_change":"2023-11-24T06:27:41.384707+0000","last_active":"2023-11-24T06:27:43.656745+0000","last_peered":"2023-11-24T06:27:43.656745+0000","last_clean":"2023-11-24T06:27:43.656745+0000","last_became_active":"2023-11-24T06:27:41.384397+0000","last_became_peered":"2023-11-24T06:27:41.384397+0000","last_unstale":"2023-11-24T06:27:43.656745+0000","last_undegraded":"2023-11-24T06:27:43.656745+0000","last_fullsized":"2023-11-24T06:27:43.656745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659143+0000","last_change":"2023-11-24T06:27:42.394007+0000","last_active":"2023-11-24T06:27:43.659143+0000","last_peered":"2023-11-24T06:27:43.659143+0000","last_clean":"2023-11-24T06:27:43.659143+0000","last_became_active":"2023-11-24T06:27:42.393636+0000","last_became_peered":"2023-11-24T06:27:42.393636+0000","last_unstale":"2023-11-24T06:27:43.659143+0000","last_undegraded":"2023-11-24T06:27:43.659143+0000","last_fullsized":"2023-11-24T06:27:43.659143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:05.800693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656767+0000","last_change":"2023-11-24T06:27:41.375368+0000","last_active":"2023-11-24T06:27:43.656767+0000","last_peered":"2023-11-24T06:27:43.656767+0000","last_clean":"2023-11-24T06:27:43.656767+0000","last_became_active":"2023-11-24T06:27:41.375062+0000","last_became_peered":"2023-11-24T06:27:41.375062+0000","last_unstale":"2023-11-24T06:27:43.656767+0000","last_undegraded":"2023-11-24T06:27:43.656767+0000","last_fullsized":"2023-11-24T06:27:43.656767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657273+0000","last_change":"2023-11-24T06:27:42.375816+0000","last_active":"2023-11-24T06:27:43.657273+0000","last_peered":"2023-11-24T06:27:43.657273+0000","last_clean":"2023-11-24T06:27:43.657273+0000","last_became_active":"2023-11-24T06:27:42.375209+0000","last_became_peered":"2023-11-24T06:27:42.375209+0000","last_unstale":"2023-11-24T06:27:43.657273+0000","last_undegraded":"2023-11-24T06:27:43.657273+0000","last_fullsized":"2023-11-24T06:27:43.657273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356462+0000","last_change":"2023-11-24T06:27:41.376512+0000","last_active":"2023-11-24T06:27:43.356462+0000","last_peered":"2023-11-24T06:27:43.356462+0000","last_clean":"2023-11-24T06:27:43.356462+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356462+0000","last_undegraded":"2023-11-24T06:27:43.356462+0000","last_fullsized":"2023-11-24T06:27:43.356462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358784+0000","last_change":"2023-11-24T06:27:42.395282+0000","last_active":"2023-11-24T06:27:43.358784+0000","last_peered":"2023-11-24T06:27:43.358784+0000","last_clean":"2023-11-24T06:27:43.358784+0000","last_became_active":"2023-11-24T06:27:42.394947+0000","last_became_peered":"2023-11-24T06:27:42.394947+0000","last_unstale":"2023-11-24T06:27:43.358784+0000","last_undegraded":"2023-11-24T06:27:43.358784+0000","last_fullsized":"2023-11-24T06:27:43.358784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:03.529167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658293+0000","last_change":"2023-11-24T06:27:41.377691+0000","last_active":"2023-11-24T06:27:43.658293+0000","last_peered":"2023-11-24T06:27:43.658293+0000","last_clean":"2023-11-24T06:27:43.658293+0000","last_became_active":"2023-11-24T06:27:41.377407+0000","last_became_peered":"2023-11-24T06:27:41.377407+0000","last_unstale":"2023-11-24T06:27:43.658293+0000","last_undegraded":"2023-11-24T06:27:43.658293+0000","last_fullsized":"2023-11-24T06:27:43.658293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:31.799834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358354+0000","last_change":"2023-11-24T06:27:42.388531+0000","last_active":"2023-11-24T06:27:43.358354+0000","last_peered":"2023-11-24T06:27:43.358354+0000","last_clean":"2023-11-24T06:27:43.358354+0000","last_became_active":"2023-11-24T06:27:42.388137+0000","last_became_peered":"2023-11-24T06:27:42.388137+0000","last_unstale":"2023-11-24T06:27:43.358354+0000","last_undegraded":"2023-11-24T06:27:43.358354+0000","last_fullsized":"2023-11-24T06:27:43.358354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658823+0000","last_change":"2023-11-24T06:27:41.383929+0000","last_active":"2023-11-24T06:27:43.658823+0000","last_peered":"2023-11-24T06:27:43.658823+0000","last_clean":"2023-11-24T06:27:43.658823+0000","last_became_active":"2023-11-24T06:27:41.383652+0000","last_became_peered":"2023-11-24T06:27:41.383652+0000","last_unstale":"2023-11-24T06:27:43.658823+0000","last_undegraded":"2023-11-24T06:27:43.658823+0000","last_fullsized":"2023-11-24T06:27:43.658823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658415+0000","last_change":"2023-11-24T06:27:42.378135+0000","last_active":"2023-11-24T06:27:43.658415+0000","last_peered":"2023-11-24T06:27:43.658415+0000","last_clean":"2023-11-24T06:27:43.658415+0000","last_became_active":"2023-11-24T06:27:42.377665+0000","last_became_peered":"2023-11-24T06:27:42.377665+0000","last_unstale":"2023-11-24T06:27:43.658415+0000","last_undegraded":"2023-11-24T06:27:43.658415+0000","last_fullsized":"2023-11-24T06:27:43.658415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659071+0000","last_change":"2023-11-24T06:27:41.388698+0000","last_active":"2023-11-24T06:27:43.659071+0000","last_peered":"2023-11-24T06:27:43.659071+0000","last_clean":"2023-11-24T06:27:43.659071+0000","last_became_active":"2023-11-24T06:27:41.388379+0000","last_became_peered":"2023-11-24T06:27:41.388379+0000","last_unstale":"2023-11-24T06:27:43.659071+0000","last_undegraded":"2023-11-24T06:27:43.659071+0000","last_fullsized":"2023-11-24T06:27:43.659071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356256+0000","last_change":"2023-11-24T06:27:42.387047+0000","last_active":"2023-11-24T06:27:43.356256+0000","last_peered":"2023-11-24T06:27:43.356256+0000","last_clean":"2023-11-24T06:27:43.356256+0000","last_became_active":"2023-11-24T06:27:42.386729+0000","last_became_peered":"2023-11-24T06:27:42.386729+0000","last_unstale":"2023-11-24T06:27:43.356256+0000","last_undegraded":"2023-11-24T06:27:43.356256+0000","last_fullsized":"2023-11-24T06:27:43.356256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:31.853326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658188+0000","last_change":"2023-11-24T06:27:41.379713+0000","last_active":"2023-11-24T06:27:43.658188+0000","last_peered":"2023-11-24T06:27:43.658188+0000","last_clean":"2023-11-24T06:27:43.658188+0000","last_became_active":"2023-11-24T06:27:41.379416+0000","last_became_peered":"2023-11-24T06:27:41.379416+0000","last_unstale":"2023-11-24T06:27:43.658188+0000","last_undegraded":"2023-11-24T06:27:43.658188+0000","last_fullsized":"2023-11-24T06:27:43.658188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660748+0000","last_change":"2023-11-24T06:27:42.392358+0000","last_active":"2023-11-24T06:27:43.660748+0000","last_peered":"2023-11-24T06:27:43.660748+0000","last_clean":"2023-11-24T06:27:43.660748+0000","last_became_active":"2023-11-24T06:27:42.392029+0000","last_became_peered":"2023-11-24T06:27:42.392029+0000","last_unstale":"2023-11-24T06:27:43.660748+0000","last_undegraded":"2023-11-24T06:27:43.660748+0000","last_fullsized":"2023-11-24T06:27:43.660748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656706+0000","last_change":"2023-11-24T06:27:41.384873+0000","last_active":"2023-11-24T06:27:43.656706+0000","last_peered":"2023-11-24T06:27:43.656706+0000","last_clean":"2023-11-24T06:27:43.656706+0000","last_became_active":"2023-11-24T06:27:41.384475+0000","last_became_peered":"2023-11-24T06:27:41.384475+0000","last_unstale":"2023-11-24T06:27:43.656706+0000","last_undegraded":"2023-11-24T06:27:43.656706+0000","last_fullsized":"2023-11-24T06:27:43.656706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357526+0000","last_change":"2023-11-24T06:27:42.381195+0000","last_active":"2023-11-24T06:27:43.357526+0000","last_peered":"2023-11-24T06:27:43.357526+0000","last_clean":"2023-11-24T06:27:43.357526+0000","last_became_active":"2023-11-24T06:27:42.377313+0000","last_became_peered":"2023-11-24T06:27:42.377313+0000","last_unstale":"2023-11-24T06:27:43.357526+0000","last_undegraded":"2023-11-24T06:27:43.357526+0000","last_fullsized":"2023-11-24T06:27:43.357526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:32.562262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657763+0000","last_change":"2023-11-24T06:27:41.377860+0000","last_active":"2023-11-24T06:27:43.657763+0000","last_peered":"2023-11-24T06:27:43.657763+0000","last_clean":"2023-11-24T06:27:43.657763+0000","last_became_active":"2023-11-24T06:27:41.377572+0000","last_became_peered":"2023-11-24T06:27:41.377572+0000","last_unstale":"2023-11-24T06:27:43.657763+0000","last_undegraded":"2023-11-24T06:27:43.657763+0000","last_fullsized":"2023-11-24T06:27:43.657763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:52.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355182+0000","last_change":"2023-11-24T06:27:42.385894+0000","last_active":"2023-11-24T06:27:43.355182+0000","last_peered":"2023-11-24T06:27:43.355182+0000","last_clean":"2023-11-24T06:27:43.355182+0000","last_became_active":"2023-11-24T06:27:42.385586+0000","last_became_peered":"2023-11-24T06:27:42.385586+0000","last_unstale":"2023-11-24T06:27:43.355182+0000","last_undegraded":"2023-11-24T06:27:43.355182+0000","last_fullsized":"2023-11-24T06:27:43.355182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658185+0000","last_change":"2023-11-24T06:27:41.380746+0000","last_active":"2023-11-24T06:27:43.658185+0000","last_peered":"2023-11-24T06:27:43.658185+0000","last_clean":"2023-11-24T06:27:43.658185+0000","last_became_active":"2023-11-24T06:27:41.380535+0000","last_became_peered":"2023-11-24T06:27:41.380535+0000","last_unstale":"2023-11-24T06:27:43.658185+0000","last_undegraded":"2023-11-24T06:27:43.658185+0000","last_fullsized":"2023-11-24T06:27:43.658185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:10.313348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355700+0000","last_change":"2023-11-24T06:27:42.399877+0000","last_active":"2023-11-24T06:27:43.355700+0000","last_peered":"2023-11-24T06:27:43.355700+0000","last_clean":"2023-11-24T06:27:43.355700+0000","last_became_active":"2023-11-24T06:27:42.399051+0000","last_became_peered":"2023-11-24T06:27:42.399051+0000","last_unstale":"2023-11-24T06:27:43.355700+0000","last_undegraded":"2023-11-24T06:27:43.355700+0000","last_fullsized":"2023-11-24T06:27:43.355700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:09.167666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659116+0000","last_change":"2023-11-24T06:27:41.386935+0000","last_active":"2023-11-24T06:27:43.659116+0000","last_peered":"2023-11-24T06:27:43.659116+0000","last_clean":"2023-11-24T06:27:43.659116+0000","last_became_active":"2023-11-24T06:27:41.386607+0000","last_became_peered":"2023-11-24T06:27:41.386607+0000","last_unstale":"2023-11-24T06:27:43.659116+0000","last_undegraded":"2023-11-24T06:27:43.659116+0000","last_fullsized":"2023-11-24T06:27:43.659116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659499+0000","last_change":"2023-11-24T06:27:42.382231+0000","last_active":"2023-11-24T06:27:43.659499+0000","last_peered":"2023-11-24T06:27:43.659499+0000","last_clean":"2023-11-24T06:27:43.659499+0000","last_became_active":"2023-11-24T06:27:42.381452+0000","last_became_peered":"2023-11-24T06:27:42.381452+0000","last_unstale":"2023-11-24T06:27:43.659499+0000","last_undegraded":"2023-11-24T06:27:43.659499+0000","last_fullsized":"2023-11-24T06:27:43.659499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358206+0000","last_change":"2023-11-24T06:27:42.378166+0000","last_active":"2023-11-24T06:27:43.358206+0000","last_peered":"2023-11-24T06:27:43.358206+0000","last_clean":"2023-11-24T06:27:43.358206+0000","last_became_active":"2023-11-24T06:27:42.377859+0000","last_became_peered":"2023-11-24T06:27:42.377859+0000","last_unstale":"2023-11-24T06:27:43.358206+0000","last_undegraded":"2023-11-24T06:27:43.358206+0000","last_fullsized":"2023-11-24T06:27:43.358206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657894+0000","last_change":"2023-11-24T06:27:41.386057+0000","last_active":"2023-11-24T06:27:43.657894+0000","last_peered":"2023-11-24T06:27:43.657894+0000","last_clean":"2023-11-24T06:27:43.657894+0000","last_became_active":"2023-11-24T06:27:41.385754+0000","last_became_peered":"2023-11-24T06:27:41.385754+0000","last_unstale":"2023-11-24T06:27:43.657894+0000","last_undegraded":"2023-11-24T06:27:43.657894+0000","last_fullsized":"2023-11-24T06:27:43.657894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357781+0000","last_change":"2023-11-24T06:27:42.382235+0000","last_active":"2023-11-24T06:27:43.357781+0000","last_peered":"2023-11-24T06:27:43.357781+0000","last_clean":"2023-11-24T06:27:43.357781+0000","last_became_active":"2023-11-24T06:27:42.380559+0000","last_became_peered":"2023-11-24T06:27:42.380559+0000","last_unstale":"2023-11-24T06:27:43.357781+0000","last_undegraded":"2023-11-24T06:27:43.357781+0000","last_fullsized":"2023-11-24T06:27:43.357781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658434+0000","last_change":"2023-11-24T06:27:41.385695+0000","last_active":"2023-11-24T06:27:43.658434+0000","last_peered":"2023-11-24T06:27:43.658434+0000","last_clean":"2023-11-24T06:27:43.658434+0000","last_became_active":"2023-11-24T06:27:41.385158+0000","last_became_peered":"2023-11-24T06:27:41.385158+0000","last_unstale":"2023-11-24T06:27:43.658434+0000","last_undegraded":"2023-11-24T06:27:43.658434+0000","last_fullsized":"2023-11-24T06:27:43.658434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660564+0000","last_change":"2023-11-24T06:27:42.396117+0000","last_active":"2023-11-24T06:27:43.660564+0000","last_peered":"2023-11-24T06:27:43.660564+0000","last_clean":"2023-11-24T06:27:43.660564+0000","last_became_active":"2023-11-24T06:27:42.395511+0000","last_became_peered":"2023-11-24T06:27:42.395511+0000","last_unstale":"2023-11-24T06:27:43.660564+0000","last_undegraded":"2023-11-24T06:27:43.660564+0000","last_fullsized":"2023-11-24T06:27:43.660564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356285+0000","last_change":"2023-11-24T06:27:41.373126+0000","last_active":"2023-11-24T06:27:43.356285+0000","last_peered":"2023-11-24T06:27:43.356285+0000","last_clean":"2023-11-24T06:27:43.356285+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356285+0000","last_undegraded":"2023-11-24T06:27:43.356285+0000","last_fullsized":"2023-11-24T06:27:43.356285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:05.485619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356095+0000","last_change":"2023-11-24T06:27:42.386786+0000","last_active":"2023-11-24T06:27:43.356095+0000","last_peered":"2023-11-24T06:27:43.356095+0000","last_clean":"2023-11-24T06:27:43.356095+0000","last_became_active":"2023-11-24T06:27:42.386492+0000","last_became_peered":"2023-11-24T06:27:42.386492+0000","last_unstale":"2023-11-24T06:27:43.356095+0000","last_undegraded":"2023-11-24T06:27:43.356095+0000","last_fullsized":"2023-11-24T06:27:43.356095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:36:11.379388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.661047+0000","last_change":"2023-11-24T06:27:41.378175+0000","last_active":"2023-11-24T06:27:43.661047+0000","last_peered":"2023-11-24T06:27:43.661047+0000","last_clean":"2023-11-24T06:27:43.661047+0000","last_became_active":"2023-11-24T06:27:41.376774+0000","last_became_peered":"2023-11-24T06:27:41.376774+0000","last_unstale":"2023-11-24T06:27:43.661047+0000","last_undegraded":"2023-11-24T06:27:43.661047+0000","last_fullsized":"2023-11-24T06:27:43.661047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":823296,"data_stored":669752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.282}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.607}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.853}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":72662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2440000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.508}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2949999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:47.765 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:27:47.766 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T06:27:47.766 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T06:27:47.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T06:27:47.766 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T06:27:48.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.027+0000 7fd3f52bb640 1 Processor -- start 2023-11-24T06:27:48.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.027+0000 7fd3f52bb640 1 -- start start 2023-11-24T06:27:48.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3f52bb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd3f0143c40 0x7fd3f0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3f52bb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd3f00717d0 0x7fd3f00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3f52bb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f0070e50 con 0x7fd3f00cca40 2023-11-24T06:27:48.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3f52bb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f00723c0 con 0x7fd3f00717d0 2023-11-24T06:27:48.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3f52bb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f00cce40 con 0x7fd3f0143c40 2023-11-24T06:27:48.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3f4aba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd3f00cca40 0x7fd3f015eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44742/0 (socket says 172.21.15.64:44742) 2023-11-24T06:27:48.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3f4aba640 1 -- 172.21.15.64:0/2222530512 learned_addr learned my addr 172.21.15.64:0/2222530512 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.029+0000 7fd3e77fe640 1 --2- 172.21.15.64:0/2222530512 >> v2:172.21.15.64:3300/0 conn(0x7fd3f00717d0 0x7fd3f00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.030+0000 7fd3effff640 1 --2- 172.21.15.64:0/2222530512 >> v2:172.21.15.162:3300/0 conn(0x7fd3f0143c40 0x7fd3f0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.030+0000 7fd3e77fe640 1 -- 172.21.15.64:0/2222530512 >> v2:172.21.15.162:3300/0 conn(0x7fd3f0143c40 msgr2=0x7fd3f0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.030+0000 7fd3e77fe640 1 --2- 172.21.15.64:0/2222530512 >> v2:172.21.15.162:3300/0 conn(0x7fd3f0143c40 0x7fd3f0071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.030+0000 7fd3e77fe640 1 -- 172.21.15.64:0/2222530512 >> v1:172.21.15.64:6789/0 conn(0x7fd3f00cca40 legacy=0x7fd3f015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.030+0000 7fd3e77fe640 1 -- 172.21.15.64:0/2222530512 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f006e170 con 0x7fd3f00717d0 2023-11-24T06:27:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.030+0000 7fd3ef7fe640 1 -- 172.21.15.64:0/2222530512 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235751338 0 0) 0x7fd3f0070e50 con 0x7fd3f00cca40 2023-11-24T06:27:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.031+0000 7fd3e77fe640 1 --2- 172.21.15.64:0/2222530512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f00717d0 0x7fd3f00cc500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd3f006e410 tx=0x7fd3d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310829b2d9b35c56 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.031+0000 7fd3ef7fe640 1 -- 172.21.15.64:0/2222530512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d802e070 con 0x7fd3f00717d0 2023-11-24T06:27:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.031+0000 7fd3ef7fe640 1 -- 172.21.15.64:0/2222530512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d80315a0 con 0x7fd3f00717d0 2023-11-24T06:27:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.031+0000 7fd3ef7fe640 1 -- 172.21.15.64:0/2222530512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8031930 con 0x7fd3f00717d0 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 -- 172.21.15.64:0/2222530512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f00717d0 msgr2=0x7fd3f00cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/2222530512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f00717d0 0x7fd3f00cc500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd3f006e410 tx=0x7fd3d802b840 comp rx=0 tx=0).stop 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 -- 172.21.15.64:0/2222530512 shutdown_connections 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/2222530512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f00717d0 0x7fd3f00cc500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/2222530512 >> v2:172.21.15.162:3300/0 conn(0x7fd3f0143c40 0x7fd3f0071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 -- 172.21.15.64:0/2222530512 >> 172.21.15.64:0/2222530512 conn(0x7fd3f006a800 msgr2=0x7fd3f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 -- 172.21.15.64:0/2222530512 shutdown_connections 2023-11-24T06:27:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.032+0000 7fd3f52bb640 1 -- 172.21.15.64:0/2222530512 wait complete. 2023-11-24T06:27:48.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.034+0000 7fd3f52bb640 1 Processor -- start 2023-11-24T06:27:48.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.034+0000 7fd3f52bb640 1 -- start start 2023-11-24T06:27:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f52bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f00717d0 0x7fd3f00ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f52bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd3f00cca40 0x7fd3f00aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f52bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f0143c40 0x7fd3f0159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f52bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0139d90 con 0x7fd3f0143c40 2023-11-24T06:27:48.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f52bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00a84c0 con 0x7fd3f00717d0 2023-11-24T06:27:48.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f52bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00a87a0 con 0x7fd3f00cca40 2023-11-24T06:27:48.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f4aba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f0143c40 0x7fd3f0159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f4aba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f0143c40 0x7fd3f0159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48186/0 (socket says 172.21.15.64:48186) 2023-11-24T06:27:48.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.035+0000 7fd3f4aba640 1 -- 172.21.15.64:0/1883256273 learned_addr learned my addr 172.21.15.64:0/1883256273 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:48.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3e77fe640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd3f00cca40 0x7fd3f00aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3effff640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f00717d0 0x7fd3f00ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f4aba640 1 -- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd3f00cca40 msgr2=0x7fd3f00aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f4aba640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd3f00cca40 0x7fd3f00aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f4aba640 1 -- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f00717d0 msgr2=0x7fd3f00ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f4aba640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f00717d0 0x7fd3f00ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f4aba640 1 -- 172.21.15.64:0/1883256273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d8009400 con 0x7fd3f0143c40 2023-11-24T06:27:48.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3e77fe640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd3f00cca40 0x7fd3f00aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f4aba640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f0143c40 0x7fd3f0159800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800c810 tx=0x7fd3e800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3ed7fa640 1 -- 172.21.15.64:0/1883256273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800a640 con 0x7fd3f0143c40 2023-11-24T06:27:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3ed7fa640 1 -- 172.21.15.64:0/1883256273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e800a7c0 con 0x7fd3f0143c40 2023-11-24T06:27:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3ed7fa640 1 -- 172.21.15.64:0/1883256273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800ab70 con 0x7fd3f0143c40 2023-11-24T06:27:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f013c970 con 0x7fd3f0143c40 2023-11-24T06:27:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.036+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f013cf30 con 0x7fd3f0143c40 2023-11-24T06:27:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.038+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f0067450 con 0x7fd3f0143c40 2023-11-24T06:27:48.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.038+0000 7fd3ed7fa640 1 -- 172.21.15.64:0/1883256273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3e800acd0 con 0x7fd3f0143c40 2023-11-24T06:27:48.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.040+0000 7fd3ed7fa640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd3d406ae40 0x7fd3d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.040+0000 7fd3ed7fa640 1 -- 172.21.15.64:0/1883256273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3e8088a10 con 0x7fd3f0143c40 2023-11-24T06:27:48.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.040+0000 7fd3effff640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd3d406ae40 0x7fd3d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.041+0000 7fd3effff640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd3d406ae40 0x7fd3d406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0006fd0 tx=0x7fd3e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.044+0000 7fd3ed7fa640 1 -- 172.21.15.64:0/1883256273 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3e8053b50 con 0x7fd3f0143c40 2023-11-24T06:27:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.289+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd3f00a88e0 con 0x7fd3f0143c40 2023-11-24T06:27:48.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.294+0000 7fd3ed7fa640 1 -- 172.21.15.64:0/1883256273 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171030 (secure 0 0 0) 0x7fd3e8018020 con 0x7fd3f0143c40 2023-11-24T06:27:48.299 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:48.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd3d406ae40 msgr2=0x7fd3d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd3d406ae40 0x7fd3d406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0006fd0 tx=0x7fd3e0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f0143c40 msgr2=0x7fd3f0159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f0143c40 0x7fd3f0159800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800c810 tx=0x7fd3e800ccd0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 shutdown_connections 2023-11-24T06:27:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd3d406ae40 0x7fd3d406d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f0143c40 0x7fd3f0159800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd3f00cca40 0x7fd3f00aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 --2- 172.21.15.64:0/1883256273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f00717d0 0x7fd3f00ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 >> 172.21.15.64:0/1883256273 conn(0x7fd3f006a800 msgr2=0x7fd3f00cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 shutdown_connections 2023-11-24T06:27:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.299+0000 7fd3f52bb640 1 -- 172.21.15.64:0/1883256273 wait complete. 2023-11-24T06:27:48.323 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1000290349},{"type":"v1","addr":"172.21.15.64:6833","nonce":1000290349}]},"active_addr":"172.21.15.64:6833/1000290349","active_change":"2023-11-24T06:27:27.073346+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":853547399}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":3221935172}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2415678094}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":773347189}]}]} 2023-11-24T06:27:48.326 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T06:27:48.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T06:27:48.327 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.544+0000 7f35b96a8640 1 Processor -- start 2023-11-24T06:27:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.544+0000 7f35b96a8640 1 -- start start 2023-11-24T06:27:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.545+0000 7f35b96a8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f35b413fcb0 0x7f35b4164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.545+0000 7f35b96a8640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b40cdc40 con 0x7f35b413cfa0 2023-11-24T06:27:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.545+0000 7f35b96a8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b40cdec0 con 0x7f35b415afb0 2023-11-24T06:27:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.545+0000 7f35b96a8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b40ce1a0 con 0x7f35b413fcb0 2023-11-24T06:27:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.546+0000 7f35b3fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f35b413cfa0 0x7f35b40cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44746/0 (socket says 172.21.15.64:44746) 2023-11-24T06:27:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.546+0000 7f35b3fff640 1 -- 172.21.15.64:0/2664142698 learned_addr learned my addr 172.21.15.64:0/2664142698 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.546+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438730581 0 0) 0x7f35b40cdc40 con 0x7f35b413cfa0 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.546+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35a0003520 con 0x7f35b413cfa0 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.546+0000 7f35b8ea7640 1 --2- 172.21.15.64:0/2664142698 >> v2:172.21.15.162:3300/0 conn(0x7f35b413fcb0 0x7f35b4164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.546+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1419550147 0 0) 0x7f35a0003520 con 0x7f35b413cfa0 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.546+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 >> v2:172.21.15.162:3300/0 conn(0x7f35b413fcb0 msgr2=0x7f35b4164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 --2- 172.21.15.64:0/2664142698 >> v2:172.21.15.162:3300/0 conn(0x7f35b413fcb0 0x7f35b4164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 >> v1:172.21.15.162:6789/0 conn(0x7f35b415afb0 legacy=0x7f35b413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b40ce480 con 0x7f35b413cfa0 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f35a4001670 con 0x7f35b413cfa0 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 >> v1:172.21.15.64:6789/0 conn(0x7f35b413cfa0 legacy=0x7f35b40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35a00047c0 0x7f35a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35a00052a0 0x7f35a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40ce1a0 con 0x7f35a0003970 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a000c170 con 0x7f35a00047c0 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a000c330 con 0x7f35a00052a0 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b3fff640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b3fff640 1 -- 172.21.15.64:0/2664142698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35a00052a0 msgr2=0x7f35a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b3fff640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35a00052a0 0x7f35a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b3fff640 1 -- 172.21.15.64:0/2664142698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35a00047c0 msgr2=0x7f35a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b3fff640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35a00047c0 0x7f35a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b3fff640 1 -- 172.21.15.64:0/2664142698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a0003520 con 0x7f35a0003970 2023-11-24T06:27:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.547+0000 7f35b8ea7640 1 -- 172.21.15.64:0/2664142698 reap_dead start 2023-11-24T06:27:48.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.548+0000 7f35b3fff640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35a0003d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c210 tx=0x7f35a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae8e0a8f247c91b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4005ad0 con 0x7f35a0003970 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4005c50 con 0x7f35a0003970 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b37fe640 1 -- 172.21.15.64:0/2664142698 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a402c870 con 0x7f35a0003970 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b96a8640 1 -- 172.21.15.64:0/2664142698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 msgr2=0x7f35a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b96a8640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35a0003d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c210 tx=0x7f35a00057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b96a8640 1 -- 172.21.15.64:0/2664142698 shutdown_connections 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b96a8640 1 --2- 172.21.15.64:0/2664142698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35a0003d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b96a8640 1 -- 172.21.15.64:0/2664142698 >> 172.21.15.64:0/2664142698 conn(0x7f35b406a800 msgr2=0x7f35b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b96a8640 1 -- 172.21.15.64:0/2664142698 shutdown_connections 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.549+0000 7f35b96a8640 1 -- 172.21.15.64:0/2664142698 wait complete. 2023-11-24T06:27:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.551+0000 7f35b96a8640 1 Processor -- start 2023-11-24T06:27:48.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.551+0000 7f35b96a8640 1 -- start start 2023-11-24T06:27:48.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.552+0000 7f35b96a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35b413c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.552+0000 7f35b96a8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35b40b3360 0x7f35b41f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.552+0000 7f35b96a8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35b4154c50 0x7f35b415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.552+0000 7f35b96a8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40adcb0 con 0x7f35a0003970 2023-11-24T06:27:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.552+0000 7f35b96a8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40ad980 con 0x7f35b40b3360 2023-11-24T06:27:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.552+0000 7f35b96a8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b4155210 con 0x7f35b4154c50 2023-11-24T06:27:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35b413c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35b413c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48198/0 (socket says 172.21.15.64:48198) 2023-11-24T06:27:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 -- 172.21.15.64:0/921311587 learned_addr learned my addr 172.21.15.64:0/921311587 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b8ea7640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35b4154c50 0x7f35b415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 -- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35b4154c50 msgr2=0x7f35b415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35b4154c50 0x7f35b415c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35ab7fe640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35b40b3360 0x7f35b41f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 -- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35b40b3360 msgr2=0x7f35b41f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35b40b3360 0x7f35b41f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 -- 172.21.15.64:0/921311587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b41554f0 con 0x7f35a0003970 2023-11-24T06:27:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35ab7fe640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35b40b3360 0x7f35b41f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.553+0000 7f35b3fff640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35b413c6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35a402c410 tx=0x7f35a402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.554+0000 7f35b17fa640 1 -- 172.21.15.64:0/921311587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a403c040 con 0x7f35a0003970 2023-11-24T06:27:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.554+0000 7f35b17fa640 1 -- 172.21.15.64:0/921311587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a402cc80 con 0x7f35a0003970 2023-11-24T06:27:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.554+0000 7f35b17fa640 1 -- 172.21.15.64:0/921311587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4007070 con 0x7f35a0003970 2023-11-24T06:27:48.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.554+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b40af620 con 0x7f35a0003970 2023-11-24T06:27:48.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.554+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b41f3800 con 0x7f35a0003970 2023-11-24T06:27:48.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.555+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b40adaf0 con 0x7f35a0003970 2023-11-24T06:27:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.556+0000 7f35b17fa640 1 -- 172.21.15.64:0/921311587 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35a4032070 con 0x7f35a0003970 2023-11-24T06:27:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.557+0000 7f35b17fa640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f359406ae70 0x7f359406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.557+0000 7f35b17fa640 1 -- 172.21.15.64:0/921311587 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35a40aa400 con 0x7f35a0003970 2023-11-24T06:27:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.557+0000 7f35ab7fe640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f359406ae70 0x7f359406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.558+0000 7f35ab7fe640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f359406ae70 0x7f359406d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35b40b3ea0 tx=0x7f359c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.561+0000 7f35b17fa640 1 -- 172.21.15.64:0/921311587 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35b40adaf0 con 0x7f35a0003970 2023-11-24T06:27:48.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.758+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35b40adaf0 con 0x7f35a0003970 2023-11-24T06:27:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.760+0000 7f35b17fa640 1 -- 172.21.15.64:0/921311587 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f35b40adaf0 con 0x7f35a0003970 2023-11-24T06:27:48.764 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:48.764 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f359406ae70 msgr2=0x7f359406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f359406ae70 0x7f359406d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35b40b3ea0 tx=0x7f359c009040 comp rx=0 tx=0).stop 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 msgr2=0x7f35b413c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35b413c6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35a402c410 tx=0x7f35a402e070 comp rx=0 tx=0).stop 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 shutdown_connections 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f359406ae70 0x7f359406d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35b4154c50 0x7f35b415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35b40b3360 0x7f35b41f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 --2- 172.21.15.64:0/921311587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a0003970 0x7f35b413c6f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 >> 172.21.15.64:0/921311587 conn(0x7f35b406a800 msgr2=0x7f35b4070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 shutdown_connections 2023-11-24T06:27:48.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:48.763+0000 7f35b96a8640 1 -- 172.21.15.64:0/921311587 wait complete. 2023-11-24T06:27:48.786 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T06:27:48.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:49.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.005+0000 7f1803147640 1 Processor -- start 2023-11-24T06:27:49.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.005+0000 7f1803147640 1 -- start start 2023-11-24T06:27:49.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.007+0000 7f1803147640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f17fc0cc990 0x7f17fc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.007+0000 7f1803147640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc070e50 con 0x7f17fc0cc500 2023-11-24T06:27:49.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.007+0000 7f1803147640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc144020 con 0x7f17fc143c40 2023-11-24T06:27:49.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.007+0000 7f1803147640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc0723c0 con 0x7f17fc0cc990 2023-11-24T06:27:49.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.007+0000 7f1801944640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f17fc0cc500 0x7f17fc06d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44758/0 (socket says 172.21.15.64:44758) 2023-11-24T06:27:49.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.007+0000 7f1801944640 1 -- 172.21.15.64:0/1903133213 learned_addr learned my addr 172.21.15.64:0/1903133213 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:49.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909135974 0 0) 0x7f17fc070e50 con 0x7f17fc0cc500 2023-11-24T06:27:49.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1802946640 1 --2- 172.21.15.64:0/1903133213 >> v2:172.21.15.162:3300/0 conn(0x7f17fc0cc990 0x7f17fc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17e8003520 con 0x7f17fc0cc500 2023-11-24T06:27:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151137319 0 0) 0x7f17e8003520 con 0x7f17fc0cc500 2023-11-24T06:27:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 >> v2:172.21.15.162:3300/0 conn(0x7f17fc0cc990 msgr2=0x7f17fc06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 --2- 172.21.15.64:0/1903133213 >> v2:172.21.15.162:3300/0 conn(0x7f17fc0cc990 0x7f17fc06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 >> v1:172.21.15.162:6789/0 conn(0x7f17fc143c40 legacy=0x7f17fc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc071e00 con 0x7f17fc0cc500 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f17ec001670 con 0x7f17fc0cc500 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 >> v1:172.21.15.64:6789/0 conn(0x7f17fc0cc500 legacy=0x7f17fc06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17e80039c0 0x7f17e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80047c0 0x7f17e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17e80051d0 0x7f17e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0723c0 con 0x7f17e80051d0 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e800c120 con 0x7f17e80039c0 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.008+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e800c2e0 con 0x7f17e80047c0 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.009+0000 7f1802946640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17e80051d0 0x7f17e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.009+0000 7f1801944640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80047c0 0x7f17e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.009+0000 7f1802946640 1 -- 172.21.15.64:0/1903133213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80047c0 msgr2=0x7f17e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.009+0000 7f1802946640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80047c0 0x7f17e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.009+0000 7f1802946640 1 -- 172.21.15.64:0/1903133213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17e80039c0 msgr2=0x7f17e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.009+0000 7f1802946640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17e80039c0 0x7f17e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.009+0000 7f1802946640 1 -- 172.21.15.64:0/1903133213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8003520 con 0x7f17e80051d0 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.010+0000 7f1802946640 1 -- 172.21.15.64:0/1903133213 reap_dead start 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.010+0000 7f1802946640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17e80051d0 0x7f17e800b980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009f90 tx=0x7f17e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eddd52f10375ff5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f402cbf0 con 0x7f17e80051d0 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f400f040 con 0x7f17e80051d0 2023-11-24T06:27:49.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1801143640 1 -- 172.21.15.64:0/1903133213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4003990 con 0x7f17e80051d0 2023-11-24T06:27:49.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1803147640 1 -- 172.21.15.64:0/1903133213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17e80051d0 msgr2=0x7f17e800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1803147640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17e80051d0 0x7f17e800b980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009f90 tx=0x7f17e80057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1803147640 1 -- 172.21.15.64:0/1903133213 shutdown_connections 2023-11-24T06:27:49.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1803147640 1 --2- 172.21.15.64:0/1903133213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17e80051d0 0x7f17e800b980 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1803147640 1 -- 172.21.15.64:0/1903133213 >> 172.21.15.64:0/1903133213 conn(0x7f17fc06a800 msgr2=0x7f17fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1803147640 1 -- 172.21.15.64:0/1903133213 shutdown_connections 2023-11-24T06:27:49.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.011+0000 7f1803147640 1 -- 172.21.15.64:0/1903133213 wait complete. 2023-11-24T06:27:49.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.013+0000 7f1803147640 1 Processor -- start 2023-11-24T06:27:49.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.013+0000 7f1803147640 1 -- start start 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.014+0000 7f1803147640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80051d0 0x7f17fc141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.014+0000 7f1803147640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17fc1593f0 0x7f17fc159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.014+0000 7f1803147640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17fc15cde0 0x7f17fc15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.014+0000 7f1803147640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc142ec0 con 0x7f17fc15cde0 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.014+0000 7f1803147640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc1419c0 con 0x7f17fc1593f0 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.014+0000 7f1803147640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc159db0 con 0x7f17e80051d0 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17fc15cde0 0x7f17fc15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17fc15cde0 0x7f17fc15d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48214/0 (socket says 172.21.15.64:48214) 2023-11-24T06:27:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 -- 172.21.15.64:0/166909215 learned_addr learned my addr 172.21.15.64:0/166909215 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802145640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80051d0 0x7f17fc141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1801944640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17fc1593f0 0x7f17fc159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 -- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80051d0 msgr2=0x7f17fc141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80051d0 0x7f17fc141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 -- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17fc1593f0 msgr2=0x7f17fc159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17fc1593f0 0x7f17fc159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 -- 172.21.15.64:0/166909215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc142340 con 0x7f17fc15cde0 2023-11-24T06:27:49.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.015+0000 7f1802946640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17fc15cde0 0x7f17fc15d260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17f402c660 tx=0x7f17f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.016+0000 7f17f2ffd640 1 -- 172.21.15.64:0/166909215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004230 con 0x7f17fc15cde0 2023-11-24T06:27:49.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.016+0000 7f17f2ffd640 1 -- 172.21.15.64:0/166909215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f4004390 con 0x7f17fc15cde0 2023-11-24T06:27:49.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.016+0000 7f17f2ffd640 1 -- 172.21.15.64:0/166909215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004740 con 0x7f17fc15cde0 2023-11-24T06:27:49.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.016+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc142620 con 0x7f17fc15cde0 2023-11-24T06:27:49.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.016+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc142be0 con 0x7f17fc15cde0 2023-11-24T06:27:49.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.017+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc067450 con 0x7f17fc15cde0 2023-11-24T06:27:49.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.018+0000 7f17f2ffd640 1 -- 172.21.15.64:0/166909215 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17f403f470 con 0x7f17fc15cde0 2023-11-24T06:27:49.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.019+0000 7f17f2ffd640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f17e006ae70 0x7f17e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.019+0000 7f17f2ffd640 1 -- 172.21.15.64:0/166909215 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17f40abe50 con 0x7f17fc15cde0 2023-11-24T06:27:49.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.019+0000 7f1802145640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f17e006ae70 0x7f17e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.020+0000 7f1802145640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f17e006ae70 0x7f17e006d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17f8001680 tx=0x7f17f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.024+0000 7f17f2ffd640 1 -- 172.21.15.64:0/166909215 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17f4076fa0 con 0x7f17fc15cde0 2023-11-24T06:27:49.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.221+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17fc0ceb30 con 0x7f17fc15cde0 2023-11-24T06:27:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.223+0000 7f17f2ffd640 1 -- 172.21.15.64:0/166909215 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f17f407a5c0 con 0x7f17fc15cde0 2023-11-24T06:27:49.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:49.228 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:49.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f17e006ae70 msgr2=0x7f17e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f17e006ae70 0x7f17e006d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17f8001680 tx=0x7f17f8009040 comp rx=0 tx=0).stop 2023-11-24T06:27:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17fc15cde0 msgr2=0x7f17fc15d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17fc15cde0 0x7f17fc15d260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17f402c660 tx=0x7f17f4009750 comp rx=0 tx=0).stop 2023-11-24T06:27:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 shutdown_connections 2023-11-24T06:27:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f17e006ae70 0x7f17e006d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17fc15cde0 0x7f17fc15d260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17fc1593f0 0x7f17fc159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 --2- 172.21.15.64:0/166909215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17e80051d0 0x7f17fc141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.226+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 >> 172.21.15.64:0/166909215 conn(0x7f17fc06a800 msgr2=0x7f17fc143220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.227+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 shutdown_connections 2023-11-24T06:27:49.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.227+0000 7f1803147640 1 -- 172.21.15.64:0/166909215 wait complete. 2023-11-24T06:27:49.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T06:27:49.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T06:27:49.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T06:27:49.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T06:27:49.252 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T06:27:49.252 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T06:27:49.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T06:27:49.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T06:27:49.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.567+0000 7fb22ead0640 1 Processor -- start 2023-11-24T06:27:49.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.567+0000 7fb22ead0640 1 -- start start 2023-11-24T06:27:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.569+0000 7fb22ead0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb228161d30 0x7fb2280b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.569+0000 7fb22ead0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2280b3a50 0x7fb228162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.569+0000 7fb22ead0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb228161b90 con 0x7fb22806d3b0 2023-11-24T06:27:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.569+0000 7fb22ead0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2280702c0 con 0x7fb228161d30 2023-11-24T06:27:49.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.569+0000 7fb22ead0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb228070840 con 0x7fb2280b3a50 2023-11-24T06:27:49.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.570+0000 7f7015101640 1 Processor -- start 2023-11-24T06:27:49.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.570+0000 7fb22c86c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb228161d30 0x7fb2280b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.570+0000 7fb22c86c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb228161d30 0x7fb2280b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48226/0 (socket says 172.21.15.64:48226) 2023-11-24T06:27:49.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.570+0000 7fb22c86c640 1 -- 172.21.15.64:0/491319405 learned_addr learned my addr 172.21.15.64:0/491319405 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.570+0000 7f7015101640 1 -- start start 2023-11-24T06:27:49.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.570+0000 7fb21ffff640 1 --2- 172.21.15.64:0/491319405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2280b3a50 0x7fb228162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.571+0000 7fb22c86c640 1 -- 172.21.15.64:0/491319405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2280b3a50 msgr2=0x7fb228162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.571+0000 7f7015101640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f701015ce90 0x7f701015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.571+0000 7f7015101640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701006c470 0x7f701006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7f9ac2c02640 1 Processor -- start 2023-11-24T06:27:49.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.571+0000 7f7015101640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7010153d30 con 0x7f7010153950 2023-11-24T06:27:49.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7f7015101640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701015d860 con 0x7f701015ce90 2023-11-24T06:27:49.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7f7015101640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70100b2b70 con 0x7f701006c470 2023-11-24T06:27:49.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7fb22c86c640 1 --2- 172.21.15.64:0/491319405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2280b3a50 0x7fb228162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7fb22c86c640 1 -- 172.21.15.64:0/491319405 >> v1:172.21.15.162:6789/0 conn(0x7fb22806d3b0 legacy=0x7fb228070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7fb22c86c640 1 -- 172.21.15.64:0/491319405 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228162e80 con 0x7fb228161d30 2023-11-24T06:27:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7f700f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701006c470 0x7f701006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7f700f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701006c470 0x7f701006c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:32810/0 (socket says 172.21.15.64:32810) 2023-11-24T06:27:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7f700f59e640 1 -- 172.21.15.64:0/2809034279 learned_addr learned my addr 172.21.15.64:0/2809034279 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.573+0000 7f700e59c640 1 --2- 172.21.15.64:0/2809034279 >> v2:172.21.15.162:3300/0 conn(0x7f701015ce90 0x7f701015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.572+0000 7f9ac2c02640 1 -- start start 2023-11-24T06:27:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7fb21ffff640 1 --2- 172.21.15.64:0/491319405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2280b3a50 0x7fb228162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f9ac2c02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f9ac2c02640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9abc0b45c0 0x7f9abc0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f9ac2c02640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9abc0b4070 0x7f9abc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f9ac2c02640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9abc0b3ea0 con 0x7f9abc0b45c0 2023-11-24T06:27:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f9ac2c02640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9abc0d1210 con 0x7f9abc0b4070 2023-11-24T06:27:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f9ac2c02640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc0b3bc0 con 0x7f9abc160c70 2023-11-24T06:27:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7fb22c86c640 1 --2- 172.21.15.64:0/491319405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb228161d30 0x7fb2280b3510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb218009e50 tx=0x7fb21802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5b595c6f24eb4d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f700f59e640 1 -- 172.21.15.64:0/2809034279 >> v2:172.21.15.162:3300/0 conn(0x7f701015ce90 msgr2=0x7f701015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f700f59e640 1 --2- 172.21.15.64:0/2809034279 >> v2:172.21.15.162:3300/0 conn(0x7f701015ce90 0x7f701015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f700f59e640 1 -- 172.21.15.64:0/2809034279 >> v1:172.21.15.162:6789/0 conn(0x7f7010153950 legacy=0x7f70100cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.574+0000 7f700f59e640 1 -- 172.21.15.64:0/2809034279 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70100b2e50 con 0x7f701006c470 2023-11-24T06:27:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7f9ac099e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7fb227fff640 1 -- 172.21.15.64:0/491319405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21802ca80 con 0x7fb228161d30 2023-11-24T06:27:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7fb227fff640 1 -- 172.21.15.64:0/491319405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21802cbe0 con 0x7fb228161d30 2023-11-24T06:27:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7fb227fff640 1 -- 172.21.15.64:0/491319405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180343f0 con 0x7fb228161d30 2023-11-24T06:27:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7f9ac099e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:32818/0 (socket says 172.21.15.64:32818) 2023-11-24T06:27:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7f9ac099e640 1 -- 172.21.15.64:0/254126362 learned_addr learned my addr 172.21.15.64:0/254126362 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7f9ac119f640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.162:3300/0 conn(0x7f9abc0b4070 0x7f9abc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.575+0000 7f13f6294640 1 Processor -- start 2023-11-24T06:27:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f13f6294640 1 -- start start 2023-11-24T06:27:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f9abbfff640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.64:3300/0 conn(0x7f9abc0b45c0 0x7f9abc0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f9ac099e640 1 -- 172.21.15.64:0/254126362 >> v2:172.21.15.162:3300/0 conn(0x7f9abc0b4070 msgr2=0x7f9abc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f9ac099e640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.162:3300/0 conn(0x7f9abc0b4070 0x7f9abc0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f9ac099e640 1 -- 172.21.15.64:0/254126362 >> v2:172.21.15.64:3300/0 conn(0x7f9abc0b45c0 msgr2=0x7f9abc0cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f9ac099e640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.64:3300/0 conn(0x7f9abc0b45c0 0x7f9abc0cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f9ac099e640 1 -- 172.21.15.64:0/254126362 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc0d0f30 con 0x7f9abc160c70 2023-11-24T06:27:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f9abbfff640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.64:3300/0 conn(0x7f9abc0b45c0 0x7f9abc0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.577+0000 7fb22ead0640 1 -- 172.21.15.64:0/491319405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb228161d30 msgr2=0x7fb2280b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.577+0000 7fb22ead0640 1 --2- 172.21.15.64:0/491319405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb228161d30 0x7fb2280b3510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb218009e50 tx=0x7fb21802bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.576+0000 7f700f59e640 1 --2- 172.21.15.64:0/2809034279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701006c470 0x7f701006c8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f700800d370 tx=0x7f700802e300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c271340222987d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.577+0000 7f9ac119f640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.162:3300/0 conn(0x7f9abc0b4070 0x7f9abc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:49.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.577+0000 7f700dd9b640 1 -- 172.21.15.64:0/2809034279 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008038600 con 0x7f701006c470 2023-11-24T06:27:49.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7fb22ead0640 1 -- 172.21.15.64:0/491319405 shutdown_connections 2023-11-24T06:27:49.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7fb22ead0640 1 --2- 172.21.15.64:0/491319405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2280b3a50 0x7fb228162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7fb22ead0640 1 --2- 172.21.15.64:0/491319405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb228161d30 0x7fb2280b3510 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7fb22ead0640 1 -- 172.21.15.64:0/491319405 >> 172.21.15.64:0/491319405 conn(0x7fb2280b0be0 msgr2=0x7fb2280b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f700dd9b640 1 -- 172.21.15.64:0/2809034279 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7008032070 con 0x7f701006c470 2023-11-24T06:27:49.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f700dd9b640 1 -- 172.21.15.64:0/2809034279 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008038980 con 0x7f701006c470 2023-11-24T06:27:49.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.577+0000 7fd07ec32640 1 Processor -- start 2023-11-24T06:27:49.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7fd07ec32640 1 -- start start 2023-11-24T06:27:49.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7fb22ead0640 1 -- 172.21.15.64:0/491319405 shutdown_connections 2023-11-24T06:27:49.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f13f6294640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13f00ce8f0 0x7f13f00ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f13f6294640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13f00b2a00 0x7f13f00b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f13f6294640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f00d0de0 con 0x7f13f0160f80 2023-11-24T06:27:49.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f13f6294640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f00d10c0 con 0x7f13f00b2a00 2023-11-24T06:27:49.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f13f6294640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f00d13a0 con 0x7f13f00ce8f0 2023-11-24T06:27:49.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7fb22ead0640 1 -- 172.21.15.64:0/491319405 wait complete. 2023-11-24T06:27:49.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f7015101640 1 -- 172.21.15.64:0/2809034279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701006c470 msgr2=0x7f701006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.578+0000 7f7015101640 1 --2- 172.21.15.64:0/2809034279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701006c470 0x7f701006c8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f700800d370 tx=0x7f700802e300 comp rx=0 tx=0).stop 2023-11-24T06:27:49.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13f4831640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13f00b2a00 0x7f13f00b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13f4831640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13f00b2a00 0x7f13f00b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48246/0 (socket says 172.21.15.64:48246) 2023-11-24T06:27:49.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f7015101640 1 -- 172.21.15.64:0/2809034279 shutdown_connections 2023-11-24T06:27:49.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f7015101640 1 --2- 172.21.15.64:0/2809034279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701006c470 0x7f701006c8e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13f4831640 1 -- 172.21.15.64:0/2519000967 learned_addr learned my addr 172.21.15.64:0/2519000967 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f7015101640 1 --2- 172.21.15.64:0/2809034279 >> v2:172.21.15.162:3300/0 conn(0x7f701015ce90 0x7f701015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f7015101640 1 -- 172.21.15.64:0/2809034279 >> 172.21.15.64:0/2809034279 conn(0x7f70100b0be0 msgr2=0x7f70100b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/2519000967 >> v2:172.21.15.162:3300/0 conn(0x7f13f00ce8f0 0x7f13f00ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13f4831640 1 -- 172.21.15.64:0/2519000967 >> v2:172.21.15.162:3300/0 conn(0x7f13f00ce8f0 msgr2=0x7f13f00ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13f4831640 1 --2- 172.21.15.64:0/2519000967 >> v2:172.21.15.162:3300/0 conn(0x7f13f00ce8f0 0x7f13f00ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13f4831640 1 -- 172.21.15.64:0/2519000967 >> v1:172.21.15.162:6789/0 conn(0x7f13f0160f80 legacy=0x7f13f0161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.579+0000 7f13f4831640 1 -- 172.21.15.64:0/2519000967 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f00d1680 con 0x7f13f00b2a00 2023-11-24T06:27:49.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7fd07ec32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078160920 0x7fd0780b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7fd07ec32640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd0780cf9f0 0x7fd0780cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7fd07ec32640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780d0a10 con 0x7fd078161010 2023-11-24T06:27:49.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7fd07ec32640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780d03b0 con 0x7fd0780cf9f0 2023-11-24T06:27:49.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7fd07ec32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780d1f50 con 0x7fd078160920 2023-11-24T06:27:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7fd07dc30640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078160920 0x7fd0780b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7fd07dc30640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078160920 0x7fd0780b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:32824/0 (socket says 172.21.15.64:32824) 2023-11-24T06:27:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7fd07dc30640 1 -- 172.21.15.64:0/2712547292 learned_addr learned my addr 172.21.15.64:0/2712547292 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7fd07d42f640 1 --2- 172.21.15.64:0/2712547292 >> v2:172.21.15.64:3300/0 conn(0x7fd0780cf9f0 0x7fd0780cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7f7015101640 1 -- 172.21.15.64:0/2809034279 shutdown_connections 2023-11-24T06:27:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7f9ac099e640 1 --2- 172.21.15.64:0/254126362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0cfa30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0b4a80 tx=0x7f9ab002b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4f127c3d511e383 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7f9abb7fe640 1 -- 172.21.15.64:0/254126362 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00317f0 con 0x7f9abc160c70 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7f9abb7fe640 1 -- 172.21.15.64:0/254126362 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0031950 con 0x7f9abc160c70 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.580+0000 7f9abb7fe640 1 -- 172.21.15.64:0/254126362 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0031ce0 con 0x7f9abc160c70 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f9ac2c02640 1 -- 172.21.15.64:0/254126362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 msgr2=0x7f9abc0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f9ac2c02640 1 --2- 172.21.15.64:0/254126362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0cfa30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0b4a80 tx=0x7f9ab002b890 comp rx=0 tx=0).stop 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f9ac2c02640 1 -- 172.21.15.64:0/254126362 shutdown_connections 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f9ac2c02640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.162:3300/0 conn(0x7f9abc0b4070 0x7f9abc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f9ac2c02640 1 --2- 172.21.15.64:0/254126362 >> v2:172.21.15.64:3300/0 conn(0x7f9abc0b45c0 0x7f9abc0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f9ac2c02640 1 --2- 172.21.15.64:0/254126362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0cfa30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f9ac2c02640 1 -- 172.21.15.64:0/254126362 >> 172.21.15.64:0/254126362 conn(0x7f9abc0d3020 msgr2=0x7f9abc0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f13f4831640 1 --2- 172.21.15.64:0/2519000967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 0x7f13f00b2e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13f00b5d10 tx=0x7f13e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c32894da2e01021 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7fb22ead0640 1 Processor -- start 2023-11-24T06:27:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7f13eeffd640 1 -- 172.21.15.64:0/2519000967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4031870 con 0x7f13f00b2a00 2023-11-24T06:27:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7f13eeffd640 1 -- 172.21.15.64:0/2519000967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e40319d0 con 0x7f13f00b2a00 2023-11-24T06:27:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7f13eeffd640 1 -- 172.21.15.64:0/2519000967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4031d60 con 0x7f13f00b2a00 2023-11-24T06:27:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7fd07d42f640 1 -- 172.21.15.64:0/2712547292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078160920 msgr2=0x7fd0780b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7fd07d42f640 1 --2- 172.21.15.64:0/2712547292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078160920 0x7fd0780b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7fd07d42f640 1 -- 172.21.15.64:0/2712547292 >> v1:172.21.15.64:6789/0 conn(0x7fd078161010 legacy=0x7fd0781614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.582+0000 7fd07d42f640 1 -- 172.21.15.64:0/2712547292 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780cf080 con 0x7fd0780cf9f0 2023-11-24T06:27:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.583+0000 7f9ac2c02640 1 -- 172.21.15.64:0/254126362 shutdown_connections 2023-11-24T06:27:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.583+0000 7fb22ead0640 1 -- start start 2023-11-24T06:27:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.581+0000 7f7015101640 1 -- 172.21.15.64:0/2809034279 wait complete. 2023-11-24T06:27:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7f7015101640 1 Processor -- start 2023-11-24T06:27:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.585+0000 7f7015101640 1 -- start start 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.583+0000 7f9ac2c02640 1 -- 172.21.15.64:0/254126362 wait complete. 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.583+0000 7f13f6294640 1 -- 172.21.15.64:0/2519000967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 msgr2=0x7f13f00b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.583+0000 7f13f6294640 1 --2- 172.21.15.64:0/2519000967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 0x7f13f00b2e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13f00b5d10 tx=0x7f13e402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7f13f6294640 1 -- 172.21.15.64:0/2519000967 shutdown_connections 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7f13f6294640 1 --2- 172.21.15.64:0/2519000967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 0x7f13f00b2e90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7f13f6294640 1 --2- 172.21.15.64:0/2519000967 >> v2:172.21.15.162:3300/0 conn(0x7f13f00ce8f0 0x7f13f00ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7f13f6294640 1 -- 172.21.15.64:0/2519000967 >> 172.21.15.64:0/2519000967 conn(0x7f13f009c550 msgr2=0x7f13f00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7f13f6294640 1 -- 172.21.15.64:0/2519000967 shutdown_connections 2023-11-24T06:27:49.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7f13f6294640 1 -- 172.21.15.64:0/2519000967 wait complete. 2023-11-24T06:27:49.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7fd07d42f640 1 --2- 172.21.15.64:0/2712547292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 0x7fd0780cfe70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0640097a0 tx=0x7fd06402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d17828c70a2366 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.584+0000 7fd07cc2e640 1 -- 172.21.15.64:0/2712547292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06402d040 con 0x7fd0780cf9f0 2023-11-24T06:27:49.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.585+0000 7fd07cc2e640 1 -- 172.21.15.64:0/2712547292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064031810 con 0x7fd0780cf9f0 2023-11-24T06:27:49.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.585+0000 7fd07cc2e640 1 -- 172.21.15.64:0/2712547292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064031ba0 con 0x7fd0780cf9f0 2023-11-24T06:27:49.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.585+0000 7f9ac2c02640 1 Processor -- start 2023-11-24T06:27:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.586+0000 7fd07ec32640 1 -- 172.21.15.64:0/2712547292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 msgr2=0x7fd0780cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.586+0000 7fd07ec32640 1 --2- 172.21.15.64:0/2712547292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 0x7fd0780cfe70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0640097a0 tx=0x7fd06402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.586+0000 7f9ac2c02640 1 -- start start 2023-11-24T06:27:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.586+0000 7fd07ec32640 1 -- 172.21.15.64:0/2712547292 shutdown_connections 2023-11-24T06:27:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.586+0000 7fd07ec32640 1 --2- 172.21.15.64:0/2712547292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 0x7fd0780cfe70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.586+0000 7fd07ec32640 1 --2- 172.21.15.64:0/2712547292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078160920 0x7fd0780b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.586+0000 7fd07ec32640 1 -- 172.21.15.64:0/2712547292 >> 172.21.15.64:0/2712547292 conn(0x7fd0780d3090 msgr2=0x7fd0780d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7fd07ec32640 1 -- 172.21.15.64:0/2712547292 shutdown_connections 2023-11-24T06:27:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7fd35b9e3640 1 Processor -- start 2023-11-24T06:27:49.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7f7015101640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701006c470 0x7f70101eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7f7015101640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7010153950 0x7f70101f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7f9ac2c02640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9abc0b4070 0x7f9abc0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7fd35b9e3640 1 -- start start 2023-11-24T06:27:49.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7f7015101640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701015ce90 0x7f701015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7f13f6294640 1 Processor -- start 2023-11-24T06:27:49.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7f7015101640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70101eefb0 con 0x7f701006c470 2023-11-24T06:27:49.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f7015101640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7010155630 con 0x7f7010153950 2023-11-24T06:27:49.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac2c02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9abc0b45c0 0x7f9abc14d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f700e59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7010153950 0x7f70101f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f700e59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7010153950 0x7f70101f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39450/0 (socket says 172.21.15.64:39450) 2023-11-24T06:27:49.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f700e59c640 1 -- 172.21.15.64:0/277088349 learned_addr learned my addr 172.21.15.64:0/277088349 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7f51853dc640 1 Processor -- start 2023-11-24T06:27:49.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f51853dc640 1 -- start start 2023-11-24T06:27:49.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7010155910 con 0x7f701015ce90 2023-11-24T06:27:49.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f700f59e640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701015ce90 0x7f701015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac2c02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac099e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9abc0b4070 0x7f9abc0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f13f6294640 1 -- start start 2023-11-24T06:27:49.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac099e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9abc0b4070 0x7f9abc0cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39456/0 (socket says 172.21.15.64:39456) 2023-11-24T06:27:49.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac099e640 1 -- 172.21.15.64:0/2321877430 learned_addr learned my addr 172.21.15.64:0/2321877430 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac2c02640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc156ad0 con 0x7f9abc0b45c0 2023-11-24T06:27:49.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f700ed9d640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701006c470 0x7f70101eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9abbfff640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9abc0b45c0 0x7f9abc14d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc0cb390 con 0x7f9abc0b4070 2023-11-24T06:27:49.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc0b5c20 con 0x7f9abc160c70 2023-11-24T06:27:49.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7fb22ead0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb22806d3b0 0x7fb2280a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.588+0000 7fb22ead0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2280b3a50 0x7fb22815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7f9ac119f640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.587+0000 7fd07ec32640 1 -- 172.21.15.64:0/2712547292 wait complete. 2023-11-24T06:27:49.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7f700ed9d640 1 -- 172.21.15.64:0/277088349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701015ce90 msgr2=0x7f701015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2280b3a50 0x7fb22815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fd07ec32640 1 Processor -- start 2023-11-24T06:27:49.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb22c86c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb22806d3b0 0x7fb2280a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb22ead0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb228161d30 0x7fb228156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb22ead0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22815a850 con 0x7fb2280b3a50 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb22ead0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228157f00 con 0x7fb228161d30 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb22ead0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281575d0 con 0x7fb22806d3b0 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb22c86c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb22806d3b0 0x7fb2280a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:32852/0 (socket says 172.21.15.64:32852) 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7fb22c86c640 1 -- 172.21.15.64:0/912690300 learned_addr learned my addr 172.21.15.64:0/912690300 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7f700ed9d640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f701015ce90 0x7f701015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7f700ed9d640 1 -- 172.21.15.64:0/277088349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7010153950 msgr2=0x7f70101f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.589+0000 7f700ed9d640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7010153950 0x7f70101f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f700ed9d640 1 -- 172.21.15.64:0/277088349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700800d050 con 0x7f701006c470 2023-11-24T06:27:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7fd07ec32640 1 -- start start 2023-11-24T06:27:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f9ac099e640 1 -- 172.21.15.64:0/2321877430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 msgr2=0x7f9abc0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f9ac099e640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9abc160c70 0x7f9abc0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f9ac099e640 1 -- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9abc0b45c0 msgr2=0x7f9abc14d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f9ac099e640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9abc0b45c0 0x7f9abc14d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f9ac099e640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4009a70 con 0x7f9abc0b4070 2023-11-24T06:27:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f51853dc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5180160f80 0x7f5180161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f51853dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51800ce8f0 0x7f51800ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f51853dc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51801618a0 con 0x7f51800cfbe0 2023-11-24T06:27:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f51853dc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800d0de0 con 0x7f51800ce8f0 2023-11-24T06:27:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f51853dc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800d10c0 con 0x7f5180160f80 2023-11-24T06:27:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f13f6294640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 0x7f13f00c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f13f6294640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13f00ce8f0 0x7f13f015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f13f6294640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f0160f80 0x7f13f00b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7fb22c86c640 1 -- 172.21.15.64:0/912690300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb228161d30 msgr2=0x7fb228156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7fb22c86c640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb228161d30 0x7fb228156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7fb22c86c640 1 -- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2280b3a50 msgr2=0x7fb22815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7fb22c86c640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2280b3a50 0x7fb22815a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7fb22c86c640 1 -- 172.21.15.64:0/912690300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210009400 con 0x7fb22806d3b0 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f13f6294640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015dc00 con 0x7f13f00b2a00 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f13f6294640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00c8150 con 0x7f13f0160f80 2023-11-24T06:27:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.590+0000 7f13f6294640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00c83f0 con 0x7f13f00ce8f0 2023-11-24T06:27:49.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7f700ed9d640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701006c470 0x7f70101eea70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f700000c370 tx=0x7f700000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fb22c86c640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb22806d3b0 0x7fb2280a8ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb218036040 tx=0x7fb218004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fd07ec32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 0x7fd0781ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fd35b9e3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd35413df80 0x7fd35415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fd35b9e3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35413e360 0x7fd3540cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fd35b9e3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35406c470 con 0x7fd35415f9c0 2023-11-24T06:27:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fd35b9e3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35406c750 con 0x7fd35413df80 2023-11-24T06:27:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fd35b9e3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35406ca30 con 0x7fd35413e360 2023-11-24T06:27:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7f13f4831640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f0160f80 0x7f13f00b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7fd07ec32640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078160920 0x7fd07815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7f517effd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5180160f80 0x7f5180161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7f517f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f51800cfbe0 0x7f51800d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44772/0 (socket says 172.21.15.64:44772) 2023-11-24T06:27:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7f517f7fe640 1 -- 172.21.15.64:0/2262348220 learned_addr learned my addr 172.21.15.64:0/2262348220 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07ec32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078161010 0x7fd07815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07ec32640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780b7590 con 0x7fd0780cf9f0 2023-11-24T06:27:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07ec32640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780b8a70 con 0x7fd078160920 2023-11-24T06:27:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07ec32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780b6cf0 con 0x7fd078161010 2023-11-24T06:27:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd358f7e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35413e360 0x7fd3540cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f13f4831640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f0160f80 0x7f13f00b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39484/0 (socket says 172.21.15.64:39484) 2023-11-24T06:27:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07d42f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078160920 0x7fd07815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f13f4831640 1 -- 172.21.15.64:0/4274347174 learned_addr learned my addr 172.21.15.64:0/4274347174 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000052d0 con 0x7f701006c470 2023-11-24T06:27:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13f00ce8f0 0x7f13f015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f13effff640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 0x7f13f00c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07d42f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078160920 0x7fd07815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39498/0 (socket says 172.21.15.64:39498) 2023-11-24T06:27:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07d42f640 1 -- 172.21.15.64:0/2114995775 learned_addr learned my addr 172.21.15.64:0/2114995775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd358f7e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35413e360 0x7fd3540cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:32870/0 (socket says 172.21.15.64:32870) 2023-11-24T06:27:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd358f7e640 1 -- 172.21.15.64:0/3137079377 learned_addr learned my addr 172.21.15.64:0/3137079377 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7fd07dc30640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 0x7fd0781ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7f9ac099e640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9abc0b4070 0x7f9abc0cae50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00090b0 tx=0x7f9ab00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.591+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0032890 con 0x7f9abc0b4070 2023-11-24T06:27:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab00094d0 con 0x7f9abc0b4070 2023-11-24T06:27:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab00329f0 con 0x7f9abc0b4070 2023-11-24T06:27:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0032d80 con 0x7f9abc0b4070 2023-11-24T06:27:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9abc1561a0 con 0x7f9abc0b4070 2023-11-24T06:27:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21803e450 con 0x7fb22806d3b0 2023-11-24T06:27:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21803e5d0 con 0x7fb22806d3b0 2023-11-24T06:27:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21803e980 con 0x7fb22806d3b0 2023-11-24T06:27:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fd07dc30640 1 -- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078161010 msgr2=0x7fd07815ebf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:27:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fd07dc30640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd078161010 0x7fd07815ebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fd07dc30640 1 -- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078160920 msgr2=0x7fd07815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fd07dc30640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078160920 0x7fd07815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fd07dc30640 1 -- 172.21.15.64:0/2114995775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064009400 con 0x7fd0780cf9f0 2023-11-24T06:27:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fd358f7e640 1 -- 172.21.15.64:0/3137079377 >> v2:172.21.15.162:3300/0 conn(0x7fd35413df80 msgr2=0x7fd35415bee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7010155bf0 con 0x7f701006c470 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70100c5100 con 0x7f701006c470 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f13f4831640 1 -- 172.21.15.64:0/4274347174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13f00ce8f0 msgr2=0x7f13f015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f13f4831640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13f00ce8f0 0x7f13f015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f13f4831640 1 -- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 msgr2=0x7f13f00c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f13f4831640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f00b2a00 0x7f13f00c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f13f4831640 1 -- 172.21.15.64:0/4274347174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e40094d0 con 0x7f13f0160f80 2023-11-24T06:27:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7f13f4831640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f0160f80 0x7f13f00b6840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13e402b950 tx=0x7f13e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd07dc30640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 0x7fd0781ff5c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00efc0 tx=0x7fd06c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fd347fff640 1 -- 172.21.15.64:0/3137079377 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665407880 0 0) 0x7fd35406c470 con 0x7fd35415f9c0 2023-11-24T06:27:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00cda0 con 0x7fd0780cf9f0 2023-11-24T06:27:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c00f040 con 0x7fd0780cf9f0 2023-11-24T06:27:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c014680 con 0x7fd0780cf9f0 2023-11-24T06:27:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f517effd640 1 -- 172.21.15.64:0/2262348220 >> v2:172.21.15.64:3300/0 conn(0x7f51800ce8f0 msgr2=0x7f51800ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f517effd640 1 --2- 172.21.15.64:0/2262348220 >> v2:172.21.15.64:3300/0 conn(0x7f51800ce8f0 0x7f51800ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f517effd640 1 -- 172.21.15.64:0/2262348220 >> v1:172.21.15.64:6789/0 conn(0x7f51800cfbe0 legacy=0x7f51800d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.592+0000 7f517effd640 1 -- 172.21.15.64:0/2262348220 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800d13a0 con 0x7f5180160f80 2023-11-24T06:27:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7000005430 con 0x7f701006c470 2023-11-24T06:27:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000003850 con 0x7f701006c470 2023-11-24T06:27:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e403acf0 con 0x7f13f0160f80 2023-11-24T06:27:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780b6fd0 con 0x7fd0780cf9f0 2023-11-24T06:27:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.595+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07815c6a0 con 0x7fd0780cf9f0 2023-11-24T06:27:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb218009b30 con 0x7fb22806d3b0 2023-11-24T06:27:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.595+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e403ae50 con 0x7f13f0160f80 2023-11-24T06:27:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.595+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4044630 con 0x7f13f0160f80 2023-11-24T06:27:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb228157b90 con 0x7fb22806d3b0 2023-11-24T06:27:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.595+0000 7fd1b1c32640 1 Processor -- start 2023-11-24T06:27:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.595+0000 7fd1b1c32640 1 -- start start 2023-11-24T06:27:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.595+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ff0000fc0 con 0x7f701006c470 2023-11-24T06:27:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.596+0000 7f517effd640 1 --2- 172.21.15.64:0/2262348220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5180160f80 0x7f5180161360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f518008d490 tx=0x7f516802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725cb654001b6dd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.593+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9abc067b60 con 0x7f9abc0b4070 2023-11-24T06:27:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd358f7e640 1 --2- 172.21.15.64:0/3137079377 >> v2:172.21.15.162:3300/0 conn(0x7fd35413df80 0x7fd35415bee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd358f7e640 1 -- 172.21.15.64:0/3137079377 >> v1:172.21.15.162:6789/0 conn(0x7fd35415f9c0 legacy=0x7fd3541408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.594+0000 7fd358f7e640 1 -- 172.21.15.64:0/3137079377 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35406cd10 con 0x7fd35413e360 2023-11-24T06:27:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.596+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f00c86d0 con 0x7f13f0160f80 2023-11-24T06:27:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.596+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb20c000fc0 con 0x7fb22806d3b0 2023-11-24T06:27:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.596+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f00b8ef0 con 0x7f13f0160f80 2023-11-24T06:27:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7f517e7fc640 1 -- 172.21.15.64:0/2262348220 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168031640 con 0x7f5180160f80 2023-11-24T06:27:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7f517e7fc640 1 -- 172.21.15.64:0/2262348220 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51680317a0 con 0x7f5180160f80 2023-11-24T06:27:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7f517e7fc640 1 -- 172.21.15.64:0/2262348220 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168031b70 con 0x7f5180160f80 2023-11-24T06:27:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0780ceac0 con 0x7fd0780cf9f0 2023-11-24T06:27:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7fd358f7e640 1 --2- 172.21.15.64:0/3137079377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35413e360 0x7fd3540cf290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd348009870 tx=0x7fd34802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab5dbc5680846b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7fd1b1c32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 0x7fd1ac0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7fd1b1c32640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd1ac0b2a00 0x7fd1ac0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7fd1b1c32640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0d0de0 con 0x7fd1ac160f80 2023-11-24T06:27:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7fd1b1c32640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac0d10c0 con 0x7fd1ac0b2a00 2023-11-24T06:27:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7fd1b1c32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0d13a0 con 0x7fd1ac0ce8f0 2023-11-24T06:27:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.597+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ab004d3f0 con 0x7f9abc0b4070 2023-11-24T06:27:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7f51853dc640 1 -- 172.21.15.64:0/2262348220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5180160f80 msgr2=0x7f5180161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7f51853dc640 1 --2- 172.21.15.64:0/2262348220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5180160f80 0x7f5180161360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f518008d490 tx=0x7f516802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7fd1ab7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd1ac160f80 0x7fd1ac161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44788/0 (socket says 172.21.15.64:44788) 2023-11-24T06:27:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7fd1ab7fe640 1 -- 172.21.15.64:0/3047646264 learned_addr learned my addr 172.21.15.64:0/3047646264 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70000039b0 con 0x7f701006c470 2023-11-24T06:27:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13f00d1c40 con 0x7f13f0160f80 2023-11-24T06:27:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7fd347fff640 1 -- 172.21.15.64:0/3137079377 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348030880 con 0x7fd35413e360 2023-11-24T06:27:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7fd347fff640 1 -- 172.21.15.64:0/3137079377 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3480309e0 con 0x7fd35413e360 2023-11-24T06:27:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.598+0000 7fd1aaffd640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 0x7fd1ac0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd347fff640 1 -- 172.21.15.64:0/3137079377 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348030880 con 0x7fd35413e360 2023-11-24T06:27:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704039696 0 0) 0x7fd1ac0d0de0 con 0x7fd1ac160f80 2023-11-24T06:27:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd198003520 con 0x7fd1ac160f80 2023-11-24T06:27:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7f9ab97fa640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9aa006ae40 0x7f9aa006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1abfff640 1 --2- 172.21.15.64:0/3047646264 >> v2:172.21.15.162:3300/0 conn(0x7fd1ac0b2a00 0x7fd1ac0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123316458 0 0) 0x7fd198003520 con 0x7fd1ac160f80 2023-11-24T06:27:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 msgr2=0x7fd1ac0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 0x7fd1ac0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 >> v2:172.21.15.162:3300/0 conn(0x7fd1ac0b2a00 msgr2=0x7fd1ac0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 --2- 172.21.15.64:0/3047646264 >> v2:172.21.15.162:3300/0 conn(0x7fd1ac0b2a00 0x7fd1ac0b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0d1680 con 0x7fd1ac160f80 2023-11-24T06:27:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab00ab6a0 con 0x7f9abc0b4070 2023-11-24T06:27:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7f6ff77fe640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6ffc06ae40 0x7f6ffc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fd194001670 con 0x7fd1ac160f80 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1abfff640 1 --2- 172.21.15.64:0/3047646264 >> v2:172.21.15.162:3300/0 conn(0x7fd1ac0b2a00 0x7fd1ac0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.599+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 >> v1:172.21.15.64:6789/0 conn(0x7fd1ac160f80 legacy=0x7fd1ac161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd1aa7fc640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd198003970 0x7fd198003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd1aa7fc640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1980047c0 0x7fd198004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd1aa7fc640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 0x7fd198005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0d10c0 con 0x7fd198003970 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0d13a0 con 0x7fd1980047c0 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19800bab0 con 0x7fd1ac0ce8f0 2023-11-24T06:27:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f700e59c640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6ffc06ae40 0x7f6ffc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f9abbfff640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9aa006ae40 0x7f9aa006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f9ab97fa640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9aa0073060 0x7f9aa0075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9aa0075b30 con 0x7f9aa0073060 2023-11-24T06:27:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab00aba50 con 0x7f9abc0b4070 2023-11-24T06:27:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f51853dc640 1 -- 172.21.15.64:0/2262348220 shutdown_connections 2023-11-24T06:27:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f51853dc640 1 --2- 172.21.15.64:0/2262348220 >> v2:172.21.15.64:3300/0 conn(0x7f51800ce8f0 0x7f51800ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f51853dc640 1 --2- 172.21.15.64:0/2262348220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5180160f80 0x7f5180161360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f51853dc640 1 -- 172.21.15.64:0/2262348220 >> 172.21.15.64:0/2262348220 conn(0x7f518009c550 msgr2=0x7f51800d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f51853dc640 1 -- 172.21.15.64:0/2262348220 shutdown_connections 2023-11-24T06:27:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7f9abbfff640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9aa006ae40 0x7f9aa006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0c8c30 tx=0x7f9aa8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd06c0147e0 con 0x7fd0780cf9f0 2023-11-24T06:27:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7f9ac119f640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9aa0073060 0x7f9aa0075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd35b9e3640 1 -- 172.21.15.64:0/3137079377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35413e360 msgr2=0x7fd3540cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13e4044790 con 0x7f13f0160f80 2023-11-24T06:27:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb21802cd50 con 0x7fb22806d3b0 2023-11-24T06:27:49.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd35b9e3640 1 --2- 172.21.15.64:0/3137079377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35413e360 0x7fd3540cf290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd348009870 tx=0x7fd34802b9c0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fb225ffb640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb20806af40 0x7fb20806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2180ac3a0 con 0x7fb22806d3b0 2023-11-24T06:27:49.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fb225ffb640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7fb208073160 0x7fb208075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7fd1abfff640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 0x7fd198005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd1ab7fe640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd198003970 0x7fd198003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd1ab7fe640 1 -- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 msgr2=0x7fd198005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd1ab7fe640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac0ce8f0 0x7fd198005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd1ab7fe640 1 -- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1980047c0 msgr2=0x7fd198004c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:49.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd1ab7fe640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1980047c0 0x7fd198004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.601+0000 7fd1ab7fe640 1 -- 172.21.15.64:0/3047646264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd198003520 con 0x7fd198003970 2023-11-24T06:27:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fd35b9e3640 1 -- 172.21.15.64:0/3137079377 shutdown_connections 2023-11-24T06:27:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.600+0000 7f51853dc640 1 -- 172.21.15.64:0/2262348220 wait complete. 2023-11-24T06:27:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 --> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb208075c30 con 0x7fb208073160 2023-11-24T06:27:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2180ac750 con 0x7fb22806d3b0 2023-11-24T06:27:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fb21ffff640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb20806af40 0x7fb20806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7f9ac119f640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9aa0073060 0x7f9aa0075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fb22d06d640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7fb208073160 0x7fb208075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7000089220 con 0x7f701006c470 2023-11-24T06:27:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fd35b9e3640 1 --2- 172.21.15.64:0/3137079377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35413e360 0x7fd3540cf290 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fd35b9e3640 1 --2- 172.21.15.64:0/3137079377 >> v2:172.21.15.162:3300/0 conn(0x7fd35413df80 0x7fd35415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7f700e59c640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6ffc06ae40 0x7f6ffc06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70101550f0 tx=0x7f7004009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.602+0000 7fd35b9e3640 1 -- 172.21.15.64:0/3137079377 >> 172.21.15.64:0/3137079377 conn(0x7fd35406a800 msgr2=0x7fd35406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fd35b9e3640 1 -- 172.21.15.64:0/3137079377 shutdown_connections 2023-11-24T06:27:49.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fd1ab7fe640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd198003970 0x7fd198003d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd19400c0e0 tx=0x7fd1980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843703405e9303a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fd0767fc640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd05c06af10 0x7fd05c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194005ad0 con 0x7fd198003970 2023-11-24T06:27:49.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fb22d06d640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7fb208073160 0x7fb208075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fd35b9e3640 1 -- 172.21.15.64:0/3137079377 wait complete. 2023-11-24T06:27:49.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7f6ff77fe640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f6ffc073060 0x7f6ffc075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 --> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ffc075b30 con 0x7f6ffc073060 2023-11-24T06:27:49.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70000895d0 con 0x7f701006c470 2023-11-24T06:27:49.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== osd.2 v2:172.21.15.64:6816/1175692500 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9aa0075b30 con 0x7f9aa0073060 2023-11-24T06:27:49.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f13ecff9640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f13dc06ae40 0x7f13dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd06c08a780 con 0x7fd0780cf9f0 2023-11-24T06:27:49.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7fd0767fc640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fd05c073130 0x7fd05c075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 --> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd05c075c00 con 0x7fd05c073130 2023-11-24T06:27:49.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06c08ab30 con 0x7fd0780cf9f0 2023-11-24T06:27:49.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7fd07d42f640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd05c06af10 0x7fd05c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f700f59e640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f6ffc073060 0x7f6ffc075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.603+0000 7fb21ffff640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb20806af40 0x7fb20806d400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb228159dd0 tx=0x7fb210008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd194005c30 con 0x7fd198003970 2023-11-24T06:27:49.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7fd1aa7fc640 1 -- 172.21.15.64:0/3047646264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194005ad0 con 0x7fd198003970 2023-11-24T06:27:49.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.605+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3047646264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd198003970 msgr2=0x7fd198003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13e402d040 con 0x7f13f0160f80 2023-11-24T06:27:49.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f13ecff9640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f13dc072fe0 0x7f13dc075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 --> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13dc075ab0 con 0x7f13dc072fe0 2023-11-24T06:27:49.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.604+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13e4067ea0 con 0x7f13f0160f80 2023-11-24T06:27:49.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.605+0000 7f13effff640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f13dc06ae40 0x7f13dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.605+0000 7f700f59e640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f6ffc073060 0x7f6ffc075480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.605+0000 7fd07d42f640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd05c06af10 0x7fd05c06d3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd07815f370 tx=0x7fd06403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.605+0000 7fd1b1c32640 1 --2- 172.21.15.64:0/3047646264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd198003970 0x7fd198003d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd19400c0e0 tx=0x7fd1980057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.605+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== osd.0 v2:172.21.15.64:6824/948599598 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb208075c30 con 0x7fb208073160 2023-11-24T06:27:49.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7fd1abfff640 1 -- 172.21.15.64:0/3047646264 reap_dead start 2023-11-24T06:27:49.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f13dc072fe0 0x7f13dc075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3047646264 shutdown_connections 2023-11-24T06:27:49.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3047646264 >> 172.21.15.64:0/3047646264 conn(0x7fd1ac09c550 msgr2=0x7fd1ac0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7fd07e431640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fd05c073130 0x7fd05c075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7f51853dc640 1 Processor -- start 2023-11-24T06:27:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7f13effff640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f13dc06ae40 0x7f13dc06d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13e8007bf0 tx=0x7f13e8007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3047646264 shutdown_connections 2023-11-24T06:27:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.607+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== osd.1 v2:172.21.15.64:6808/1463145862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6ffc075b30 con 0x7f6ffc073060 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.607+0000 7f51853dc640 1 -- start start 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.606+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3047646264 wait complete. 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.607+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f13dc072fe0 0x7f13dc075400 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.608+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== osd.3 v2:172.21.15.64:6800/3744810476 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f13dc075ab0 con 0x7f13dc072fe0 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.607+0000 7fd07e431640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fd05c073130 0x7fd05c075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== osd.4 v2:172.21.15.162:6802/399042930 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd05c075c00 con 0x7fd05c073130 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.607+0000 7fd35b9e3640 1 Processor -- start 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.608+0000 7fd35b9e3640 1 -- start start 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35b9e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd35413df80 0x7fd35413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35b9e3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd35413e360 0x7fd35414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35b9e3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35415f9c0 0x7fd35414de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35b9e3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35414acf0 con 0x7fd35413df80 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35b9e3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35414e9e0 con 0x7fd35413e360 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35b9e3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35414ecc0 con 0x7fd35415f9c0 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35977f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd35413df80 0x7fd35413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35977f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd35413df80 0x7fd35413d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48326/0 (socket says 172.21.15.64:48326) 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd35977f640 1 -- 172.21.15.64:0/787137158 learned_addr learned my addr 172.21.15.64:0/787137158 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd359f80640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35415f9c0 0x7fd35414de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd358f7e640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd35413e360 0x7fd35414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd35977f640 1 -- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35415f9c0 msgr2=0x7fd35414de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd35977f640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd35415f9c0 0x7fd35414de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd35977f640 1 -- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd35413e360 msgr2=0x7fd35414b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd35977f640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd35413e360 0x7fd35414b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd35977f640 1 -- 172.21.15.64:0/787137158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480094d0 con 0x7fd35413df80 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7f51853dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51800ce8f0 0x7f518015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7f51853dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51800cfbe0 0x7f5180160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7f51853dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5180160f80 0x7f51801f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f51853dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180160750 con 0x7f5180160f80 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f51853dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518015dd80 con 0x7f51800ce8f0 2023-11-24T06:27:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f51853dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518015f1b0 con 0x7f51800cfbe0 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f517effd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51800ce8f0 0x7f518015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f517effd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51800ce8f0 0x7f518015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39544/0 (socket says 172.21.15.64:39544) 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f517effd640 1 -- 172.21.15.64:0/2912804046 learned_addr learned my addr 172.21.15.64:0/2912804046 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f51767fc640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51800cfbe0 0x7f5180160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7f517f7fe640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5180160f80 0x7f51801f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.609+0000 7fd1b1c32640 1 Processor -- start 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.610+0000 7fd1b1c32640 1 -- start start 2023-11-24T06:27:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.611+0000 7fd35977f640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd35413df80 0x7fd35413d9f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd35000b3e0 tx=0x7fd35000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.611+0000 7f517effd640 1 -- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51800cfbe0 msgr2=0x7f5180160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.611+0000 7f517effd640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51800cfbe0 0x7f5180160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.611+0000 7f517effd640 1 -- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5180160f80 msgr2=0x7f51801f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.611+0000 7f517effd640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5180160f80 0x7f51801f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.611+0000 7f517effd640 1 -- 172.21.15.64:0/2912804046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168009400 con 0x7f51800ce8f0 2023-11-24T06:27:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.611+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350011830 con 0x7fd35413df80 2023-11-24T06:27:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.612+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35414a980 con 0x7fd35413df80 2023-11-24T06:27:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.612+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35414efa0 con 0x7fd35413df80 2023-11-24T06:27:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.612+0000 7f517effd640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51800ce8f0 0x7f518015d6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5168009860 tx=0x7f51680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.612+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516803bc20 con 0x7f51800ce8f0 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.612+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516803bd80 con 0x7f51800ce8f0 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.612+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518015f450 con 0x7f51800ce8f0 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.613+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518015fa10 con 0x7f51800ce8f0 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.613+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516803bc20 con 0x7f51800ce8f0 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.615+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5164000fc0 con 0x7f51800ce8f0 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.613+0000 7fd1b1c32640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd198003970 0x7fd1ac156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1b1c32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac1600b0 0x7fd1ac15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1b1c32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac15ac60 0x7fd1ac15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1b1c32640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac15a300 con 0x7fd1ac15ac60 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1b1c32640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac157000 con 0x7fd198003970 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1b1c32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0b60e0 con 0x7fd1ac1600b0 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1abfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac15ac60 0x7fd1ac15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac1600b0 0x7fd1ac15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.614+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac1600b0 0x7fd1ac15a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:32934/0 (socket says 172.21.15.64:32934) 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.615+0000 7fd1aaffd640 1 -- 172.21.15.64:0/3725881236 learned_addr learned my addr 172.21.15.64:0/3725881236 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.615+0000 7fd1ab7fe640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd198003970 0x7fd1ac156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.616+0000 7fd1abfff640 1 -- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac1600b0 msgr2=0x7fd1ac15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.616+0000 7fd1abfff640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1ac1600b0 0x7fd1ac15a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.616+0000 7fd1abfff640 1 -- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd198003970 msgr2=0x7fd1ac156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.616+0000 7fd1abfff640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd198003970 0x7fd1ac156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.616+0000 7fd1abfff640 1 -- 172.21.15.64:0/3725881236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac159680 con 0x7fd1ac15ac60 2023-11-24T06:27:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.616+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f516802f050 con 0x7f51800ce8f0 2023-11-24T06:27:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.616+0000 7fd1abfff640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac15ac60 0x7fd1ac15bc70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0cd810 tx=0x7fd1a400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.612+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd350011990 con 0x7fd35413df80 2023-11-24T06:27:49.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350011d40 con 0x7fd35413df80 2023-11-24T06:27:49.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd334000fc0 con 0x7fd35413df80 2023-11-24T06:27:49.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7f5177fff640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f516006aea0 0x7f516006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400a650 con 0x7fd1ac15ac60 2023-11-24T06:27:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51680ab820 con 0x7f51800ce8f0 2023-11-24T06:27:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7f5177fff640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7f51600730c0 0x7f51600754e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.618+0000 7f51767fc640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f516006aea0 0x7f516006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac159960 con 0x7fd1ac15ac60 2023-11-24T06:27:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.617+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac159f20 con 0x7fd1ac15ac60 2023-11-24T06:27:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.619+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 --> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5160075b90 con 0x7f51600730c0 2023-11-24T06:27:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.618+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a400a7b0 con 0x7fd1ac15ac60 2023-11-24T06:27:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.619+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400a650 con 0x7fd1ac15ac60 2023-11-24T06:27:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.619+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5168068ea0 con 0x7f51800ce8f0 2023-11-24T06:27:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.619+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd190000fc0 con 0x7fd1ac15ac60 2023-11-24T06:27:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.619+0000 7f517f7fe640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7f51600730c0 0x7f51600754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.620+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd35001a400 con 0x7fd35413df80 2023-11-24T06:27:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.620+0000 7f51767fc640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f516006aea0 0x7f516006d360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51800cd5d0 tx=0x7f5170009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.620+0000 7f517f7fe640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7f51600730c0 0x7f51600754e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.621+0000 7fd345ffb640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd33806ae40 0x7fd33806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.621+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd350089770 con 0x7fd35413df80 2023-11-24T06:27:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.622+0000 7fd345ffb640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7fd338072e70 0x7fd338075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.622+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 --> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd338075940 con 0x7fd338072e70 2023-11-24T06:27:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.623+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== osd.6 v2:172.21.15.162:6800/1044826159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5160075b90 con 0x7f51600730c0 2023-11-24T06:27:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.623+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1a400a920 con 0x7fd1ac15ac60 2023-11-24T06:27:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.624+0000 7fd1a3fff640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd18c06af00 0x7fd18c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.625+0000 7fd359f80640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7fd338072e70 0x7fd338075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.625+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1a4058c80 con 0x7fd1ac15ac60 2023-11-24T06:27:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.625+0000 7fd1a3fff640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fd18c073d00 0x7fd18c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.625+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 --> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c076870 con 0x7fd18c073d00 2023-11-24T06:27:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.625+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1a400a650 con 0x7fd1ac15ac60 2023-11-24T06:27:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.626+0000 7fd1ab7fe640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd18c06af00 0x7fd18c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.626+0000 7fd358f7e640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd33806ae40 0x7fd33806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.626+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd350011ea0 con 0x7fd35413df80 2023-11-24T06:27:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.627+0000 7fd1aaffd640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fd18c073d00 0x7fd18c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.626+0000 7fd359f80640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7fd338072e70 0x7fd338075290 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.627+0000 7fd358f7e640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd33806ae40 0x7fd33806d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd348036000 tx=0x7fd348008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.627+0000 7fd1ab7fe640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd18c06af00 0x7fd18c06d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd194034000 tx=0x7fd194007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.628+0000 7fd1aaffd640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fd18c073d00 0x7fd18c0761a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.629+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== osd.5 v2:172.21.15.162:6807/694094717 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd338075940 con 0x7fd338072e70 2023-11-24T06:27:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.630+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== osd.7 v2:172.21.15.162:6824/4255674423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd18c076870 con 0x7fd18c073d00 2023-11-24T06:27:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.637+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 --> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9abc156c10 con 0x7f9aa0073060 2023-11-24T06:27:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.637+0000 7f9ab97fa640 1 -- 172.21.15.64:0/2321877430 <== osd.2 v2:172.21.15.64:6816/1175692500 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9aa0075b30 con 0x7f9aa0073060 2023-11-24T06:27:49.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.638+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9aa0073060 msgr2=0x7f9aa0075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.638+0000 7f9ac2c02640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9aa0073060 0x7f9aa0075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.638+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9aa006ae40 msgr2=0x7f9aa006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.638+0000 7f9ac2c02640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9aa006ae40 0x7f9aa006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0c8c30 tx=0x7f9aa8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.638+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9abc0b4070 msgr2=0x7f9abc0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.638+0000 7f9ac2c02640 1 --2- 172.21.15.64:0/2321877430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9abc0b4070 0x7f9abc0cae50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00090b0 tx=0x7f9ab00046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.639+0000 7f9ac119f640 1 -- 172.21.15.64:0/2321877430 reap_dead start 2023-11-24T06:27:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.639+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 --> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb20c002cb0 con 0x7fb208073160 2023-11-24T06:27:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.639+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 shutdown_connections 2023-11-24T06:27:49.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.639+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 >> 172.21.15.64:0/2321877430 conn(0x7f9abc0d3020 msgr2=0x7f9abc0ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.640+0000 7fb225ffb640 1 -- 172.21.15.64:0/912690300 <== osd.0 v2:172.21.15.64:6824/948599598 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb20c002cb0 con 0x7fb208073160 2023-11-24T06:27:49.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.640+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 shutdown_connections 2023-11-24T06:27:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.640+0000 7f9ac2c02640 1 -- 172.21.15.64:0/2321877430 wait complete. 2023-11-24T06:27:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.644+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7fb208073160 msgr2=0x7fb208075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.644+0000 7fb22ead0640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7fb208073160 0x7fb208075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.644+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb20806af40 msgr2=0x7fb20806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.644+0000 7fb22ead0640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb20806af40 0x7fb20806d400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb228159dd0 tx=0x7fb210008040 comp rx=0 tx=0).stop 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.644+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb22806d3b0 msgr2=0x7fb2280a8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.644+0000 7fb22ead0640 1 --2- 172.21.15.64:0/912690300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb22806d3b0 0x7fb2280a8ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb218036040 tx=0x7fb218004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.645+0000 7fb22d06d640 1 -- 172.21.15.64:0/912690300 reap_dead start 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.645+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 shutdown_connections 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.645+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 >> 172.21.15.64:0/912690300 conn(0x7fb2280b0be0 msgr2=0x7fb2280b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.645+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 shutdown_connections 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.645+0000 7fb22ead0640 1 -- 172.21.15.64:0/912690300 wait complete. 2023-11-24T06:27:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.646+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 --> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6ff0002cb0 con 0x7f6ffc073060 2023-11-24T06:27:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.647+0000 7f6ff77fe640 1 -- 172.21.15.64:0/277088349 <== osd.1 v2:172.21.15.64:6808/1463145862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6ff0002cb0 con 0x7f6ffc073060 2023-11-24T06:27:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.647+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f6ffc073060 msgr2=0x7f6ffc075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.647+0000 7f7015101640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f6ffc073060 0x7f6ffc075480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.648+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6ffc06ae40 msgr2=0x7f6ffc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.648+0000 7f7015101640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6ffc06ae40 0x7f6ffc06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70101550f0 tx=0x7f7004009190 comp rx=0 tx=0).stop 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.648+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701006c470 msgr2=0x7f70101eea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.648+0000 7f7015101640 1 --2- 172.21.15.64:0/277088349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701006c470 0x7f70101eea70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f700000c370 tx=0x7f700000c830 comp rx=0 tx=0).stop 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.648+0000 7f700f59e640 1 -- 172.21.15.64:0/277088349 reap_dead start 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.648+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 shutdown_connections 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.648+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 >> 172.21.15.64:0/277088349 conn(0x7f70100b0be0 msgr2=0x7f701006d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.649+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 shutdown_connections 2023-11-24T06:27:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.649+0000 7f7015101640 1 -- 172.21.15.64:0/277088349 wait complete. 2023-11-24T06:27:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.649+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 --> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0780ce7e0 con 0x7fd05c073130 2023-11-24T06:27:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.650+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 --> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f13f00c8bf0 con 0x7f13dc072fe0 2023-11-24T06:27:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.651+0000 7f13ecff9640 1 -- 172.21.15.64:0/4274347174 <== osd.3 v2:172.21.15.64:6800/3744810476 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f13f00c8bf0 con 0x7f13dc072fe0 2023-11-24T06:27:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.651+0000 7fd0767fc640 1 -- 172.21.15.64:0/2114995775 <== osd.4 v2:172.21.15.162:6802/399042930 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd070001080 con 0x7fd05c073130 2023-11-24T06:27:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.652+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fd05c073130 msgr2=0x7fd05c075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.652+0000 7fd07ec32640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fd05c073130 0x7fd05c075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.653+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f13dc072fe0 msgr2=0x7f13dc075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.653+0000 7f13f6294640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f13dc072fe0 0x7f13dc075400 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.653+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 --> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5164002cb0 con 0x7f51600730c0 2023-11-24T06:27:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd05c06af10 msgr2=0x7fd05c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07ec32640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd05c06af10 0x7fd05c06d3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd07815f370 tx=0x7fd06403c040 comp rx=0 tx=0).stop 2023-11-24T06:27:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 msgr2=0x7fd0781ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07ec32640 1 --2- 172.21.15.64:0/2114995775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0780cf9f0 0x7fd0781ff5c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00efc0 tx=0x7fd06c00c490 comp rx=0 tx=0).stop 2023-11-24T06:27:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f13dc06ae40 msgr2=0x7f13dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7f13f6294640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f13dc06ae40 0x7f13dc06d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13e8007bf0 tx=0x7f13e8007240 comp rx=0 tx=0).stop 2023-11-24T06:27:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f0160f80 msgr2=0x7f13f00b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7f13f6294640 1 --2- 172.21.15.64:0/4274347174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13f0160f80 0x7f13f00b6840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13e402b950 tx=0x7f13e40046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07e431640 1 -- 172.21.15.64:0/2114995775 reap_dead start 2023-11-24T06:27:49.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 shutdown_connections 2023-11-24T06:27:49.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 >> 172.21.15.64:0/2114995775 conn(0x7fd0780d3090 msgr2=0x7fd0780c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.655+0000 7f5177fff640 1 -- 172.21.15.64:0/2912804046 <== osd.6 v2:172.21.15.162:6800/1044826159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5164002cb0 con 0x7f51600730c0 2023-11-24T06:27:49.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.654+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 shutdown_connections 2023-11-24T06:27:49.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.655+0000 7fd07ec32640 1 -- 172.21.15.64:0/2114995775 wait complete. 2023-11-24T06:27:49.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.655+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7f51600730c0 msgr2=0x7f51600754e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.655+0000 7f51853dc640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7f51600730c0 0x7f51600754e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f516006aea0 msgr2=0x7f516006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f13f4831640 1 -- 172.21.15.64:0/4274347174 reap_dead start 2023-11-24T06:27:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 shutdown_connections 2023-11-24T06:27:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 >> 172.21.15.64:0/4274347174 conn(0x7f13f009c550 msgr2=0x7f13f0161360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 shutdown_connections 2023-11-24T06:27:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f51853dc640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f516006aea0 0x7f516006d360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51800cd5d0 tx=0x7f5170009060 comp rx=0 tx=0).stop 2023-11-24T06:27:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f13f6294640 1 -- 172.21.15.64:0/4274347174 wait complete. 2023-11-24T06:27:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.656+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51800ce8f0 msgr2=0x7f518015d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.657+0000 7f51853dc640 1 --2- 172.21.15.64:0/2912804046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51800ce8f0 0x7f518015d6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5168009860 tx=0x7f51680046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.657+0000 7f517f7fe640 1 -- 172.21.15.64:0/2912804046 reap_dead start 2023-11-24T06:27:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.657+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 shutdown_connections 2023-11-24T06:27:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.657+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 >> 172.21.15.64:0/2912804046 conn(0x7f518009c550 msgr2=0x7f51800b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.657+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 --> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd190002cb0 con 0x7fd18c073d00 2023-11-24T06:27:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.657+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 shutdown_connections 2023-11-24T06:27:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.658+0000 7f51853dc640 1 -- 172.21.15.64:0/2912804046 wait complete. 2023-11-24T06:27:49.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.659+0000 7fd1a3fff640 1 -- 172.21.15.64:0/3725881236 <== osd.7 v2:172.21.15.162:6824/4255674423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd190002cb0 con 0x7fd18c073d00 2023-11-24T06:27:49.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.660+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fd18c073d00 msgr2=0x7fd18c0761a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.660+0000 7fd1b1c32640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fd18c073d00 0x7fd18c0761a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.661+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd18c06af00 msgr2=0x7fd18c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.661+0000 7fd1b1c32640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd18c06af00 0x7fd18c06d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd194034000 tx=0x7fd194007000 comp rx=0 tx=0).stop 2023-11-24T06:27:49.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.661+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac15ac60 msgr2=0x7fd1ac15bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.661+0000 7fd1b1c32640 1 --2- 172.21.15.64:0/3725881236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac15ac60 0x7fd1ac15bc70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0cd810 tx=0x7fd1a400c7b0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.661+0000 7fd1abfff640 1 -- 172.21.15.64:0/3725881236 reap_dead start 2023-11-24T06:27:49.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.662+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 --> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd334002cb0 con 0x7fd338072e70 2023-11-24T06:27:49.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.662+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 shutdown_connections 2023-11-24T06:27:49.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.662+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 >> 172.21.15.64:0/3725881236 conn(0x7fd1ac09c550 msgr2=0x7fd1ac06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.665+0000 7fd345ffb640 1 -- 172.21.15.64:0/787137158 <== osd.5 v2:172.21.15.162:6807/694094717 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd334002cb0 con 0x7fd338072e70 2023-11-24T06:27:49.695 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:49.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.665+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 shutdown_connections 2023-11-24T06:27:49.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.665+0000 7fd1b1c32640 1 -- 172.21.15.64:0/3725881236 wait complete. 2023-11-24T06:27:49.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:27:49.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.666+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7fd338072e70 msgr2=0x7fd338075290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.666+0000 7fd35b9e3640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7fd338072e70 0x7fd338075290 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.666+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd33806ae40 msgr2=0x7fd33806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.666+0000 7fd35b9e3640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd33806ae40 0x7fd33806d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd348036000 tx=0x7fd348008030 comp rx=0 tx=0).stop 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.666+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd35413df80 msgr2=0x7fd35413d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.666+0000 7fd35b9e3640 1 --2- 172.21.15.64:0/787137158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd35413df80 0x7fd35413d9f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd35000b3e0 tx=0x7fd35000b8a0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.667+0000 7fd359f80640 1 -- 172.21.15.64:0/787137158 reap_dead start 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.667+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 shutdown_connections 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.667+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 >> 172.21.15.64:0/787137158 conn(0x7fd35406a800 msgr2=0x7fd35415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.667+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 shutdown_connections 2023-11-24T06:27:49.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:49.667+0000 7fd35b9e3640 1 -- 172.21.15.64:0/787137158 wait complete. 2023-11-24T06:27:49.699 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-11-24T06:27:49.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:27:49.700 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:49.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:27:49.701 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:49.701 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:27:49.702 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:49.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:27:49.703 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-11-24T06:27:49.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:27:49.704 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:49.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:27:49.705 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-11-24T06:27:49.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:27:50.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.039+0000 7f8fa759e640 1 Processor -- start 2023-11-24T06:27:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.040+0000 7f8fa759e640 1 -- start start 2023-11-24T06:27:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.041+0000 7fabb9e97640 1 Processor -- start 2023-11-24T06:27:50.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.041+0000 7fabb9e97640 1 -- start start 2023-11-24T06:27:50.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.042+0000 7f8fa759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa80b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.042+0000 7f8fa759e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8fa8162810 0x7f8fa8071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.042+0000 7f8fa759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fa80b3650 0x7f8fa8071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.042+0000 7f8fa759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa8071cc0 con 0x7f8fa80b3650 2023-11-24T06:27:50.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.042+0000 7f8fa759e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa80702e0 con 0x7f8fa8162810 2023-11-24T06:27:50.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.042+0000 7f8fa759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80705c0 con 0x7f8fa8162430 2023-11-24T06:27:50.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.043+0000 7fa7b4432640 1 Processor -- start 2023-11-24T06:27:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.043+0000 7fa7b4432640 1 -- start start 2023-11-24T06:27:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.043+0000 7f8fa659c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa80b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.044+0000 7fabb9e97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb40b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.043+0000 7f8fa659c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa80b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:32938/0 (socket says 172.21.15.64:32938) 2023-11-24T06:27:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.043+0000 7f8fa659c640 1 -- 172.21.15.64:0/2774000198 learned_addr learned my addr 172.21.15.64:0/2774000198 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.044+0000 7fabb9e97640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fabb40cfad0 0x7fabb4070e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.044+0000 7f8f9ffff640 1 --2- 172.21.15.64:0/2774000198 >> v2:172.21.15.162:3300/0 conn(0x7f8fa8162810 0x7f8fa8071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fabb2ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fabb41598d0 0x7fabb40cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44798/0 (socket says 172.21.15.64:44798) 2023-11-24T06:27:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.044+0000 7f8fa6d9d640 1 --2- 172.21.15.64:0/2774000198 >> v2:172.21.15.64:3300/0 conn(0x7f8fa80b3650 0x7f8fa8071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fa7b4432640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7ac0ce740 0x7fa7ac0cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fa7b4432640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac1426e0 0x7fa7ac0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fa7b4432640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa7ac142c30 0x7fa7ac071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fa7b4432640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac0ceb20 con 0x7fa7ac142c30 2023-11-24T06:27:50.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fa7b4432640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac140410 con 0x7fa7ac0ce740 2023-11-24T06:27:50.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fa7b4432640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1406f0 con 0x7fa7ac1426e0 2023-11-24T06:27:50.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fabb3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fabb40cfad0 0x7fabb4070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fabb3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fabb40cfad0 0x7fabb4070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48380/0 (socket says 172.21.15.64:48380) 2023-11-24T06:27:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fabb37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb40b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.045+0000 7fabb9e97640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb4159e20 con 0x7fabb41598d0 2023-11-24T06:27:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fabb2ffd640 1 -- 172.21.15.64:0/911583301 learned_addr learned my addr 172.21.15.64:0/911583301 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fabb9e97640 1 -- 172.21.15.64:0/911583301 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb4153e30 con 0x7fabb40cfad0 2023-11-24T06:27:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fabb9e97640 1 -- 172.21.15.64:0/911583301 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40709b0 con 0x7fabb4153a50 2023-11-24T06:27:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fa7b29cf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa7ac142c30 0x7fa7ac071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fa7b29cf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa7ac142c30 0x7fa7ac071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48384/0 (socket says 172.21.15.64:48384) 2023-11-24T06:27:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fa7b29cf640 1 -- 172.21.15.64:0/2047636080 learned_addr learned my addr 172.21.15.64:0/2047636080 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/2047636080 >> v2:172.21.15.162:3300/0 conn(0x7fa7ac0ce740 0x7fa7ac0cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.046+0000 7fa7b19cd640 1 --2- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac1426e0 0x7fa7ac0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fabb3fff640 1 -- 172.21.15.64:0/911583301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 msgr2=0x7fabb40b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fabb3fff640 1 --2- 172.21.15.64:0/911583301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb40b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fabb3fff640 1 -- 172.21.15.64:0/911583301 >> v1:172.21.15.64:6789/0 conn(0x7fabb41598d0 legacy=0x7fabb40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fabb3fff640 1 -- 172.21.15.64:0/911583301 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb415a100 con 0x7fabb40cfad0 2023-11-24T06:27:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fa7b21ce640 1 -- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac1426e0 msgr2=0x7fa7ac0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac1426e0 0x7fa7ac0cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fa7b21ce640 1 -- 172.21.15.64:0/2047636080 >> v2:172.21.15.64:3300/0 conn(0x7fa7ac142c30 msgr2=0x7fa7ac071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/2047636080 >> v2:172.21.15.64:3300/0 conn(0x7fa7ac142c30 0x7fa7ac071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.047+0000 7fa7b21ce640 1 -- 172.21.15.64:0/2047636080 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac1409d0 con 0x7fa7ac0ce740 2023-11-24T06:27:50.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.048+0000 7f8fa659c640 1 -- 172.21.15.64:0/2774000198 >> v2:172.21.15.162:3300/0 conn(0x7f8fa8162810 msgr2=0x7f8fa8071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.048+0000 7f8fa659c640 1 --2- 172.21.15.64:0/2774000198 >> v2:172.21.15.162:3300/0 conn(0x7f8fa8162810 0x7f8fa8071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.048+0000 7f8fa659c640 1 -- 172.21.15.64:0/2774000198 >> v2:172.21.15.64:3300/0 conn(0x7f8fa80b3650 msgr2=0x7f8fa8071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.049+0000 7fabb37fe640 1 --2- 172.21.15.64:0/911583301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb40b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:50.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.049+0000 7fa7b29cf640 1 --2- 172.21.15.64:0/2047636080 >> v2:172.21.15.64:3300/0 conn(0x7fa7ac142c30 0x7fa7ac071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:50.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.049+0000 7fabb3fff640 1 --2- 172.21.15.64:0/911583301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4070e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faba800dbc0 tx=0x7faba802fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4e7cb86c84e4c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.048+0000 7f8fa659c640 1 --2- 172.21.15.64:0/2774000198 >> v2:172.21.15.64:3300/0 conn(0x7f8fa80b3650 0x7f8fa8071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.050+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac0cef40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0b53c0 tx=0x7fa79c02b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f45b48815a40c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.048+0000 7f8fa659c640 1 -- 172.21.15.64:0/2774000198 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa80708a0 con 0x7f8fa8162430 2023-11-24T06:27:50.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.050+0000 7fabb27fc640 1 -- 172.21.15.64:0/911583301 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8030910 con 0x7fabb40cfad0 2023-11-24T06:27:50.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.050+0000 7f8fa6d9d640 1 --2- 172.21.15.64:0/2774000198 >> v2:172.21.15.64:3300/0 conn(0x7f8fa80b3650 0x7f8fa8071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:50.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.050+0000 7fabb27fc640 1 -- 172.21.15.64:0/911583301 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8030a70 con 0x7fabb40cfad0 2023-11-24T06:27:50.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.050+0000 7fabb27fc640 1 -- 172.21.15.64:0/911583301 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8030e40 con 0x7fabb40cfad0 2023-11-24T06:27:50.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.050+0000 7fa7b11cc640 1 -- 172.21.15.64:0/2047636080 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c031520 con 0x7fa7ac0ce740 2023-11-24T06:27:50.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.050+0000 7f8fa659c640 1 --2- 172.21.15.64:0/2774000198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa80b3110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8fa806e440 tx=0x7f8f9002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843105dc4c2d98bb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.051+0000 7fa7b11cc640 1 -- 172.21.15.64:0/2047636080 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c031680 con 0x7fa7ac0ce740 2023-11-24T06:27:50.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.051+0000 7fa7b11cc640 1 -- 172.21.15.64:0/2047636080 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c031520 con 0x7fa7ac0ce740 2023-11-24T06:27:50.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7f8fa5d9b640 1 -- 172.21.15.64:0/2774000198 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90030800 con 0x7f8fa8162430 2023-11-24T06:27:50.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.051+0000 7fabb9e97640 1 -- 172.21.15.64:0/911583301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 msgr2=0x7fabb4070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.051+0000 7fabb9e97640 1 --2- 172.21.15.64:0/911583301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4070e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faba800dbc0 tx=0x7faba802fd70 comp rx=0 tx=0).stop 2023-11-24T06:27:50.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fabb9e97640 1 -- 172.21.15.64:0/911583301 shutdown_connections 2023-11-24T06:27:50.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fabb9e97640 1 --2- 172.21.15.64:0/911583301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4070e00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fabb9e97640 1 --2- 172.21.15.64:0/911583301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb40b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fabb9e97640 1 -- 172.21.15.64:0/911583301 >> 172.21.15.64:0/911583301 conn(0x7fabb406a800 msgr2=0x7fabb406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fabb9e97640 1 -- 172.21.15.64:0/911583301 shutdown_connections 2023-11-24T06:27:50.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7f8fa5d9b640 1 -- 172.21.15.64:0/2774000198 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f90030960 con 0x7f8fa8162430 2023-11-24T06:27:50.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7f8fa5d9b640 1 -- 172.21.15.64:0/2774000198 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90030d30 con 0x7f8fa8162430 2023-11-24T06:27:50.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fa7b4432640 1 -- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 msgr2=0x7fa7ac0cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fa7b4432640 1 --2- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac0cef40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0b53c0 tx=0x7fa79c02b7c0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fabb9e97640 1 -- 172.21.15.64:0/911583301 wait complete. 2023-11-24T06:27:50.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fa7b4432640 1 -- 172.21.15.64:0/2047636080 shutdown_connections 2023-11-24T06:27:50.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.052+0000 7fa7b4432640 1 --2- 172.21.15.64:0/2047636080 >> v2:172.21.15.64:3300/0 conn(0x7fa7ac142c30 0x7fa7ac071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.053+0000 7fa7b4432640 1 --2- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac1426e0 0x7fa7ac0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.053+0000 7fa7b4432640 1 --2- 172.21.15.64:0/2047636080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac0cef40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.053+0000 7fa7b4432640 1 -- 172.21.15.64:0/2047636080 >> 172.21.15.64:0/2047636080 conn(0x7fa7ac0b4710 msgr2=0x7fa7ac0b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.053+0000 7fa7b4432640 1 -- 172.21.15.64:0/2047636080 shutdown_connections 2023-11-24T06:27:50.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.053+0000 7f8fa759e640 1 -- 172.21.15.64:0/2774000198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 msgr2=0x7f8fa80b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.053+0000 7f8fa759e640 1 --2- 172.21.15.64:0/2774000198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa80b3110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8fa806e440 tx=0x7f8f9002b940 comp rx=0 tx=0).stop 2023-11-24T06:27:50.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.053+0000 7f8fa759e640 1 -- 172.21.15.64:0/2774000198 shutdown_connections 2023-11-24T06:27:50.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.054+0000 7f8fa759e640 1 --2- 172.21.15.64:0/2774000198 >> v2:172.21.15.64:3300/0 conn(0x7f8fa80b3650 0x7f8fa8071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.054+0000 7f8fa759e640 1 --2- 172.21.15.64:0/2774000198 >> v2:172.21.15.162:3300/0 conn(0x7f8fa8162810 0x7f8fa8071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.054+0000 7f8fa759e640 1 --2- 172.21.15.64:0/2774000198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa80b3110 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.054+0000 7f8fa759e640 1 -- 172.21.15.64:0/2774000198 >> 172.21.15.64:0/2774000198 conn(0x7f8fa806a800 msgr2=0x7f8fa806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.054+0000 7f8fa759e640 1 -- 172.21.15.64:0/2774000198 shutdown_connections 2023-11-24T06:27:50.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.054+0000 7fa7b4432640 1 -- 172.21.15.64:0/2047636080 wait complete. 2023-11-24T06:27:50.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.055+0000 7fabb9e97640 1 Processor -- start 2023-11-24T06:27:50.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.055+0000 7fabb9e97640 1 -- start start 2023-11-24T06:27:50.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.056+0000 7fa7b4432640 1 Processor -- start 2023-11-24T06:27:50.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.057+0000 7fa7b4432640 1 -- start start 2023-11-24T06:27:50.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.054+0000 7f8fa759e640 1 -- 172.21.15.64:0/2774000198 wait complete. 2023-11-24T06:27:50.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.057+0000 7fabb9e97640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.057+0000 7f8fa759e640 1 Processor -- start 2023-11-24T06:27:50.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.057+0000 7f8fa759e640 1 -- start start 2023-11-24T06:27:50.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.058+0000 7fabb37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.058+0000 7fabb37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48396/0 (socket says 172.21.15.64:48396) 2023-11-24T06:27:50.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.058+0000 7fabb37fe640 1 -- 172.21.15.64:0/223578030 learned_addr learned my addr 172.21.15.64:0/223578030 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.058+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb41f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.058+0000 7fa7b4432640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac097cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.058+0000 7fa7b4432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac1426e0 0x7fa7ac06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.058+0000 7fa7b4432640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac142c30 0x7fa7ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb41598d0 0x7fabb41f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b4432640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac159a20 con 0x7fa7ac1426e0 2023-11-24T06:27:50.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b19cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac1426e0 0x7fa7ac06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fabb2ffd640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb41f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b19cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac1426e0 0x7fa7ac06cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48410/0 (socket says 172.21.15.64:48410) 2023-11-24T06:27:50.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b19cd640 1 -- 172.21.15.64:0/4227926759 learned_addr learned my addr 172.21.15.64:0/4227926759 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac155c80 con 0x7fa7ac0ce740 2023-11-24T06:27:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac150d20 con 0x7fa7ac142c30 2023-11-24T06:27:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac097cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.059+0000 7fa7b29cf640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac142c30 0x7fa7ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb414f9f0 con 0x7fabb40cfad0 2023-11-24T06:27:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41e29c0 con 0x7fabb41598d0 2023-11-24T06:27:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41e2ca0 con 0x7fabb4153a50 2023-11-24T06:27:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7f8fa759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fa80b3650 0x7f8fa81ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fabb3fff640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb41598d0 0x7fabb41f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7f8fa759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa8156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fa7b21ce640 1 -- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac142c30 msgr2=0x7fa7ac06d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac142c30 0x7fa7ac06d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fa7b21ce640 1 -- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac1426e0 msgr2=0x7fa7ac06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac1426e0 0x7fa7ac06cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fa7b21ce640 1 -- 172.21.15.64:0/4227926759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4009b30 con 0x7fa7ac0ce740 2023-11-24T06:27:50.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.060+0000 7fa7b29cf640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac142c30 0x7fa7ac06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:50.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7fa7b19cd640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac1426e0 0x7fa7ac06cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:50.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7fabb37fe640 1 -- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 msgr2=0x7fabb41f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7fabb37fe640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb41f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7fabb37fe640 1 -- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb41598d0 msgr2=0x7fabb41f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7fabb37fe640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb41598d0 0x7fabb41f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7fabb37fe640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba800d8a0 con 0x7fabb40cfad0 2023-11-24T06:27:50.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7f8fa759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8157170 0x7f8fa8153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7ff3196ff640 1 Processor -- start 2023-11-24T06:27:50.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7ff3196ff640 1 -- start start 2023-11-24T06:27:50.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7f8fa659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fa80b3650 0x7f8fa81ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7f8f9ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa8156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.061+0000 7fa7b21ce640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac097cc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa79c033000 tx=0x7fa79c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7f8fa759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa8157620 con 0x7f8fa8157170 2023-11-24T06:27:50.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7f8fa759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa815b910 con 0x7f8fa80b3650 2023-11-24T06:27:50.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7f8fa759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa8158d60 con 0x7f8fa8162430 2023-11-24T06:27:50.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7f8fa659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fa80b3650 0x7f8fa81ff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39590/0 (socket says 172.21.15.64:39590) 2023-11-24T06:27:50.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7f8fa659c640 1 -- 172.21.15.64:0/144348322 learned_addr learned my addr 172.21.15.64:0/144348322 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7f8fa6d9d640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8157170 0x7f8fa8153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8fa6d9d640 1 -- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 msgr2=0x7f8fa8156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.062+0000 7fabb37fe640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4150bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d080 tx=0x7fabac00d540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8fa6d9d640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa8156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8fa6d9d640 1 -- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fa80b3650 msgr2=0x7f8fa81ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8fa6d9d640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fa80b3650 0x7f8fa81ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8fa6d9d640 1 -- 172.21.15.64:0/144348322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900094d0 con 0x7f8fa8157170 2023-11-24T06:27:50.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7fa7a2ffd640 1 -- 172.21.15.64:0/4227926759 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c031520 con 0x7fa7ac0ce740 2023-11-24T06:27:50.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7fa7a2ffd640 1 -- 172.21.15.64:0/4227926759 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c031ec0 con 0x7fa7ac0ce740 2023-11-24T06:27:50.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7fa7a2ffd640 1 -- 172.21.15.64:0/4227926759 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c033c50 con 0x7fa7ac0ce740 2023-11-24T06:27:50.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8fa659c640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fa80b3650 0x7f8fa81ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8fa6d9d640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8157170 0x7f8fa8153810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007f20 tx=0x7f8fa000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7f8f9ffff640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa8156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7fab9bfff640 1 -- 172.21.15.64:0/223578030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac010070 con 0x7fabb40cfad0 2023-11-24T06:27:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7ff3196ff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff30c0e3130 0x7ff30c0e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7ff3196ff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c007b40 con 0x7ff30c053c80 2023-11-24T06:27:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7ff3196ff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c0587e0 con 0x7ff30c03c070 2023-11-24T06:27:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7ff3196ff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30c058ac0 con 0x7ff30c0e3130 2023-11-24T06:27:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.063+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c009400 con 0x7fa7ac0ce740 2023-11-24T06:27:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7fab9bfff640 1 -- 172.21.15.64:0/223578030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac014430 con 0x7fabb40cfad0 2023-11-24T06:27:50.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7fab9bfff640 1 -- 172.21.15.64:0/223578030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0147e0 con 0x7fabb40cfad0 2023-11-24T06:27:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac159c70 con 0x7fa7ac0ce740 2023-11-24T06:27:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7f8f9effd640 1 -- 172.21.15.64:0/144348322 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0003b90 con 0x7f8fa8157170 2023-11-24T06:27:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa8158fd0 con 0x7f8fa8157170 2023-11-24T06:27:50.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7f8f9effd640 1 -- 172.21.15.64:0/144348322 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa000f040 con 0x7f8fa8157170 2023-11-24T06:27:50.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb41e2f80 con 0x7fabb40cfad0 2023-11-24T06:27:50.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb41e3540 con 0x7fabb40cfad0 2023-11-24T06:27:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.066+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab94005250 con 0x7fabb40cfad0 2023-11-24T06:27:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.064+0000 7f8f9effd640 1 -- 172.21.15.64:0/144348322 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0003b90 con 0x7f8fa8157170 2023-11-24T06:27:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.066+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac0675d0 con 0x7fa7ac0ce740 2023-11-24T06:27:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.065+0000 7ff318efe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff30c053c80 0x7ff30c0540d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44814/0 (socket says 172.21.15.64:44814) 2023-11-24T06:27:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.066+0000 7ff3137fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff30c03c070 0x7ff30c058020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:53964/0 (socket says 172.21.15.64:53964) 2023-11-24T06:27:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.066+0000 7ff3137fe640 1 -- 172.21.15.64:0/1274486316 learned_addr learned my addr 172.21.15.64:0/1274486316 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.067+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487824282 0 0) 0x7ff30c007b40 con 0x7ff30c053c80 2023-11-24T06:27:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.067+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff304003520 con 0x7ff30c053c80 2023-11-24T06:27:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.067+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129199385 0 0) 0x7ff30c0587e0 con 0x7ff30c03c070 2023-11-24T06:27:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.067+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff30c007b40 con 0x7ff30c03c070 2023-11-24T06:27:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.066+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa8159590 con 0x7f8fa8157170 2023-11-24T06:27:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.068+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa81597b0 con 0x7f8fa8157170 2023-11-24T06:27:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.069+0000 7ff313fff640 1 --2- 172.21.15.64:0/1274486316 >> v2:172.21.15.64:3300/0 conn(0x7ff30c0e3130 0x7ff30c0e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042633838 0 0) 0x7ff304003520 con 0x7ff30c053c80 2023-11-24T06:27:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 >> v2:172.21.15.64:3300/0 conn(0x7ff30c0e3130 msgr2=0x7ff30c0e7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 --2- 172.21.15.64:0/1274486316 >> v2:172.21.15.64:3300/0 conn(0x7ff30c0e3130 0x7ff30c0e7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 >> v1:172.21.15.162:6789/0 conn(0x7ff30c03c070 legacy=0x7ff30c058020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c006330 con 0x7ff30c053c80 2023-11-24T06:27:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7fa7a2ffd640 1 -- 172.21.15.64:0/4227926759 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa79c03a040 con 0x7fa7ac0ce740 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7ff31404f090 con 0x7ff30c053c80 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 >> v1:172.21.15.64:6789/0 conn(0x7ff30c053c80 legacy=0x7ff30c0540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff304003b00 0x7ff304003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3040047c0 0x7ff304004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c058ac0 con 0x7ff304005180 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30400bfd0 con 0x7ff3040047c0 2023-11-24T06:27:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30400c190 con 0x7ff304003b00 2023-11-24T06:27:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.070+0000 7f8f9effd640 1 -- 172.21.15.64:0/144348322 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fa0003cf0 con 0x7f8fa8157170 2023-11-24T06:27:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.071+0000 7ff318efe640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7ff318efe640 1 -- 172.21.15.64:0/1274486316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff304003b00 msgr2=0x7ff304003f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7ff318efe640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff304003b00 0x7ff304003f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7ff318efe640 1 -- 172.21.15.64:0/1274486316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3040047c0 msgr2=0x7ff304004c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:27:50.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7f8f9effd640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8f8806aed0 0x7f8f8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7ff318efe640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3040047c0 0x7ff304004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7ff318efe640 1 -- 172.21.15.64:0/1274486316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304003520 con 0x7ff304005180 2023-11-24T06:27:50.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7fa7a2ffd640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa79006af10 0x7fa79006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7fa7a2ffd640 1 -- 172.21.15.64:0/4227926759 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa79c0aba10 con 0x7fa7ac0ce740 2023-11-24T06:27:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.073+0000 7fa7b19cd640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa79006af10 0x7fa79006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7f4b466bb640 1 Processor -- start 2023-11-24T06:27:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.072+0000 7f4b466bb640 1 -- start start 2023-11-24T06:27:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.073+0000 7ff318efe640 1 -- 172.21.15.64:0/1274486316 reap_dead start 2023-11-24T06:27:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.074+0000 7ff318efe640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30400b980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff314064320 tx=0x7ff3040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7f20a3b475d8ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.074+0000 7f8fa659c640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8f8806aed0 0x7f8f8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.074+0000 7fa666683640 1 Processor -- start 2023-11-24T06:27:50.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.074+0000 7f4b466bb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b4015bd40 0x7f4b40071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.074+0000 7fa7b19cd640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa79006af10 0x7fa79006d3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac15cc50 tx=0x7fa7a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.074+0000 7f4b466bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4006d0f0 0x7f4b40071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7f4b466bb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b40162b30 con 0x7f4b40161aa0 2023-11-24T06:27:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7f4b466bb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4006ccd0 con 0x7f4b4015bd40 2023-11-24T06:27:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7f4b466bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40162640 con 0x7f4b4006d0f0 2023-11-24T06:27:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7f4b456b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b4015bd40 0x7f4b40071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7fab9bfff640 1 -- 172.21.15.64:0/223578030 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fabac014940 con 0x7fabb40cfad0 2023-11-24T06:27:50.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7fa666683640 1 -- start start 2023-11-24T06:27:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7f4b456b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b4015bd40 0x7f4b40071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48444/0 (socket says 172.21.15.64:48444) 2023-11-24T06:27:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7f4b456b9640 1 -- 172.21.15.64:0/1888567927 learned_addr learned my addr 172.21.15.64:0/1888567927 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.075+0000 7f4b44eb8640 1 --2- 172.21.15.64:0/1888567927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4006d0f0 0x7f4b40071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.076+0000 7f4b44eb8640 1 -- 172.21.15.64:0/1888567927 >> v2:172.21.15.64:3300/0 conn(0x7f4b4015bd40 msgr2=0x7f4b40071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.076+0000 7f4b44eb8640 1 --2- 172.21.15.64:0/1888567927 >> v2:172.21.15.64:3300/0 conn(0x7f4b4015bd40 0x7f4b40071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.076+0000 7f4b44eb8640 1 -- 172.21.15.64:0/1888567927 >> v1:172.21.15.64:6789/0 conn(0x7f4b40161aa0 legacy=0x7f4b40161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.076+0000 7f4b44eb8640 1 -- 172.21.15.64:0/1888567927 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b40162850 con 0x7f4b4006d0f0 2023-11-24T06:27:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.077+0000 7fab9bfff640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7faba006af10 0x7faba006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.077+0000 7fab9bfff640 1 -- 172.21.15.64:0/223578030 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabac08acb0 con 0x7fabb40cfad0 2023-11-24T06:27:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.077+0000 7fabb2ffd640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7faba006af10 0x7faba006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.077+0000 7fa666683640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa6600b4280 0x7fa6600b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.077+0000 7f8fa659c640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8f8806aed0 0x7f8f8806d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f9002b980 tx=0x7f8f90008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa666683640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b3c10 0x7fa6600713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa666683640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa66006e7e0 0x7fa66006ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa666683640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6600cdf90 con 0x7fa6600b4280 2023-11-24T06:27:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa666683640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6600b4ba0 con 0x7fa66006e7e0 2023-11-24T06:27:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa666683640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa660072000 con 0x7fa6600b3c10 2023-11-24T06:27:50.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140ae640 con 0x7ff304005180 2023-11-24T06:27:50.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fabb2ffd640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7faba006af10 0x7faba006d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabb414fd30 tx=0x7faba4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7f8f9effd640 1 -- 172.21.15.64:0/144348322 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fa00580a0 con 0x7f8fa8157170 2023-11-24T06:27:50.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa65ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa6600b4280 0x7fa6600b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa65ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa6600b4280 0x7fa6600b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48448/0 (socket says 172.21.15.64:48448) 2023-11-24T06:27:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.078+0000 7fa65ffff640 1 -- 172.21.15.64:0/1563269635 learned_addr learned my addr 172.21.15.64:0/1563269635 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fab9bfff640 1 -- 172.21.15.64:0/223578030 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fabac055e80 con 0x7fabb40cfad0 2023-11-24T06:27:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa6577fe640 1 --2- 172.21.15.64:0/1563269635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b3c10 0x7fa6600713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7f4b44eb8640 1 --2- 172.21.15.64:0/1888567927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4006d0f0 0x7f4b40071640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097a0 tx=0x7f4b3002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ce79ceee079af2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa664c20640 1 --2- 172.21.15.64:0/1563269635 >> v2:172.21.15.162:3300/0 conn(0x7fa66006e7e0 0x7fa66006ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa65ffff640 1 -- 172.21.15.64:0/1563269635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b3c10 msgr2=0x7fa6600713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa65ffff640 1 --2- 172.21.15.64:0/1563269635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b3c10 0x7fa6600713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa65ffff640 1 -- 172.21.15.64:0/1563269635 >> v2:172.21.15.162:3300/0 conn(0x7fa66006e7e0 msgr2=0x7fa66006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa65ffff640 1 --2- 172.21.15.64:0/1563269635 >> v2:172.21.15.162:3300/0 conn(0x7fa66006e7e0 0x7fa66006ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa65ffff640 1 -- 172.21.15.64:0/1563269635 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa660071d20 con 0x7fa6600b4280 2023-11-24T06:27:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7fa664c20640 1 --2- 172.21.15.64:0/1563269635 >> v2:172.21.15.162:3300/0 conn(0x7fa66006e7e0 0x7fa66006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7f4b37fff640 1 -- 172.21.15.64:0/1888567927 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3002d040 con 0x7f4b4006d0f0 2023-11-24T06:27:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7f3a67090640 1 Processor -- start 2023-11-24T06:27:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.079+0000 7f4b37fff640 1 -- 172.21.15.64:0/1888567927 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b30031810 con 0x7f4b4006d0f0 2023-11-24T06:27:50.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.080+0000 7f4b37fff640 1 -- 172.21.15.64:0/1888567927 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30031ba0 con 0x7f4b4006d0f0 2023-11-24T06:27:50.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.080+0000 7f3a67090640 1 -- start start 2023-11-24T06:27:50.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.080+0000 7f4b466bb640 1 -- 172.21.15.64:0/1888567927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4006d0f0 msgr2=0x7f4b40071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.080+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1888567927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4006d0f0 0x7f4b40071640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097a0 tx=0x7f4b3002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f3a67090640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a600b3ae0 0x7f3a6015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f3a67090640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a600cf290 0x7f3a600cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f4b466bb640 1 -- 172.21.15.64:0/1888567927 shutdown_connections 2023-11-24T06:27:50.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1888567927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4006d0f0 0x7f4b40071640 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1888567927 >> v2:172.21.15.64:3300/0 conn(0x7f4b4015bd40 0x7f4b40071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f4b466bb640 1 -- 172.21.15.64:0/1888567927 >> 172.21.15.64:0/1888567927 conn(0x7f4b400b0b70 msgr2=0x7f4b400b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7fa65ffff640 1 --2- 172.21.15.64:0/1563269635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b4280 0x7fa6600b4660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa660074960 tx=0x7fa65002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4c290c5108a630 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f3a67090640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a60070c40 con 0x7f3a600ae5a0 2023-11-24T06:27:50.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f3a67090640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a60071450 con 0x7f3a600cf290 2023-11-24T06:27:50.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f3a67090640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a600cfc60 con 0x7f3a600b3ae0 2023-11-24T06:27:50.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.081+0000 7f4b466bb640 1 -- 172.21.15.64:0/1888567927 shutdown_connections 2023-11-24T06:27:50.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a6562d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a600cf290 0x7f3a600cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a6562d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a600cf290 0x7f3a600cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48456/0 (socket says 172.21.15.64:48456) 2023-11-24T06:27:50.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a6562d640 1 -- 172.21.15.64:0/1085695142 learned_addr learned my addr 172.21.15.64:0/1085695142 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a5ffff640 1 --2- 172.21.15.64:0/1085695142 >> v2:172.21.15.162:3300/0 conn(0x7f3a600b3ae0 0x7f3a6015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a6562d640 1 -- 172.21.15.64:0/1085695142 >> v2:172.21.15.162:3300/0 conn(0x7f3a600b3ae0 msgr2=0x7f3a6015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a6562d640 1 --2- 172.21.15.64:0/1085695142 >> v2:172.21.15.162:3300/0 conn(0x7f3a600b3ae0 0x7f3a6015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a6562d640 1 -- 172.21.15.64:0/1085695142 >> v1:172.21.15.64:6789/0 conn(0x7f3a600ae5a0 legacy=0x7f3a6006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f3a6562d640 1 -- 172.21.15.64:0/1085695142 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6006c470 con 0x7f3a600cf290 2023-11-24T06:27:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.082+0000 7f4b466bb640 1 -- 172.21.15.64:0/1888567927 wait complete. 2023-11-24T06:27:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.084+0000 7f3a6562d640 1 --2- 172.21.15.64:0/1085695142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600cf290 0x7f3a600cf720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a600cfea0 tx=0x7f3a5402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf2e02b12ee3e81 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.084+0000 7f4b466bb640 1 Processor -- start 2023-11-24T06:27:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.084+0000 7f4b466bb640 1 -- start start 2023-11-24T06:27:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.085+0000 7f4b466bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b4006d0f0 0x7f4b4015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b466bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4015bd40 0x7f4b400c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b466bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b40161aa0 0x7f4b4014a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b466bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4014ae70 con 0x7f4b40161aa0 2023-11-24T06:27:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b466bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4015f680 con 0x7f4b4006d0f0 2023-11-24T06:27:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b466bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4015f960 con 0x7f4b4015bd40 2023-11-24T06:27:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.085+0000 7fa7a2ffd640 1 -- 172.21.15.64:0/4227926759 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa79c076be0 con 0x7fa7ac0ce740 2023-11-24T06:27:50.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b456b9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b4006d0f0 0x7f4b4015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b45eba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b40161aa0 0x7f4b4014a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b456b9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b4006d0f0 0x7f4b4015f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39622/0 (socket says 172.21.15.64:39622) 2023-11-24T06:27:50.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b456b9640 1 -- 172.21.15.64:0/1361421223 learned_addr learned my addr 172.21.15.64:0/1361421223 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.086+0000 7f4b44eb8640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4015bd40 0x7f4b400c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7f4b45eba640 1 -- 172.21.15.64:0/1361421223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4015bd40 msgr2=0x7f4b400c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7f4b45eba640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4015bd40 0x7f4b400c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7f4b45eba640 1 -- 172.21.15.64:0/1361421223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b4006d0f0 msgr2=0x7f4b4015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7f4b45eba640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b4006d0f0 0x7f4b4015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7f4b45eba640 1 -- 172.21.15.64:0/1361421223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b30009400 con 0x7f4b40161aa0 2023-11-24T06:27:50.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7fa65f7fe640 1 -- 172.21.15.64:0/1563269635 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65002ca80 con 0x7fa6600b4280 2023-11-24T06:27:50.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7fa65f7fe640 1 -- 172.21.15.64:0/1563269635 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65002cbe0 con 0x7fa6600b4280 2023-11-24T06:27:50.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7fa65f7fe640 1 -- 172.21.15.64:0/1563269635 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6500343f0 con 0x7fa6600b4280 2023-11-24T06:27:50.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7f3a5f7fe640 1 -- 172.21.15.64:0/1085695142 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54030860 con 0x7f3a600cf290 2023-11-24T06:27:50.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.087+0000 7f3a5f7fe640 1 -- 172.21.15.64:0/1085695142 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a540309c0 con 0x7f3a600cf290 2023-11-24T06:27:50.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f3a5f7fe640 1 -- 172.21.15.64:0/1085695142 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54030860 con 0x7f3a600cf290 2023-11-24T06:27:50.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f4b45eba640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b40161aa0 0x7f4b4014a930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800c730 tx=0x7f4b3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3140ae7a0 con 0x7ff304005180 2023-11-24T06:27:50.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7ff312ffd640 1 -- 172.21.15.64:0/1274486316 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140ae640 con 0x7ff304005180 2023-11-24T06:27:50.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f4b35ffb640 1 -- 172.21.15.64:0/1361421223 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38006d50 con 0x7f4b40161aa0 2023-11-24T06:27:50.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f8f9effd640 1 -- 172.21.15.64:0/144348322 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8fa00543e0 con 0x7f8fa8157170 2023-11-24T06:27:50.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7fa666683640 1 -- 172.21.15.64:0/1563269635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b4280 msgr2=0x7fa6600b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7fa666683640 1 --2- 172.21.15.64:0/1563269635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b4280 0x7fa6600b4660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa660074960 tx=0x7fa65002bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f4b35ffb640 1 -- 172.21.15.64:0/1361421223 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38003970 con 0x7f4b40161aa0 2023-11-24T06:27:50.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f4b35ffb640 1 -- 172.21.15.64:0/1361421223 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38003d60 con 0x7f4b40161aa0 2023-11-24T06:27:50.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f3a67090640 1 -- 172.21.15.64:0/1085695142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600cf290 msgr2=0x7f3a600cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.088+0000 7f3a67090640 1 --2- 172.21.15.64:0/1085695142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600cf290 0x7f3a600cf720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a600cfea0 tx=0x7f3a5402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4015fc40 con 0x7f4b40161aa0 2023-11-24T06:27:50.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7f3a67090640 1 -- 172.21.15.64:0/1085695142 shutdown_connections 2023-11-24T06:27:50.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7f3a67090640 1 --2- 172.21.15.64:0/1085695142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600cf290 0x7f3a600cf720 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7f3a67090640 1 --2- 172.21.15.64:0/1085695142 >> v2:172.21.15.162:3300/0 conn(0x7f3a600b3ae0 0x7f3a6015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7f3a67090640 1 -- 172.21.15.64:0/1085695142 >> 172.21.15.64:0/1085695142 conn(0x7f3a6006a800 msgr2=0x7f3a6006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7ff3196ff640 1 -- 172.21.15.64:0/1274486316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 msgr2=0x7ff30400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30400b980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff314064320 tx=0x7ff3040057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b40160200 con 0x7f4b40161aa0 2023-11-24T06:27:50.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7ff3196ff640 1 -- 172.21.15.64:0/1274486316 shutdown_connections 2023-11-24T06:27:50.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.089+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1274486316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30400b980 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.090+0000 7ff3196ff640 1 -- 172.21.15.64:0/1274486316 >> 172.21.15.64:0/1274486316 conn(0x7ff30c058eb0 msgr2=0x7ff30c059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7f3a67090640 1 -- 172.21.15.64:0/1085695142 shutdown_connections 2023-11-24T06:27:50.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7ff3196ff640 1 -- 172.21.15.64:0/1274486316 shutdown_connections 2023-11-24T06:27:50.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7fa666683640 1 -- 172.21.15.64:0/1563269635 shutdown_connections 2023-11-24T06:27:50.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7fa666683640 1 --2- 172.21.15.64:0/1563269635 >> v2:172.21.15.162:3300/0 conn(0x7fa66006e7e0 0x7fa66006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7fa666683640 1 --2- 172.21.15.64:0/1563269635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b3c10 0x7fa6600713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7fa666683640 1 --2- 172.21.15.64:0/1563269635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b4280 0x7fa6600b4660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7fa666683640 1 -- 172.21.15.64:0/1563269635 >> 172.21.15.64:0/1563269635 conn(0x7fa6600cfef0 msgr2=0x7fa6600d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7f2e6d7ff640 1 Processor -- start 2023-11-24T06:27:50.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.092+0000 7ff3196ff640 1 -- 172.21.15.64:0/1274486316 wait complete. 2023-11-24T06:27:50.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7f3a67090640 1 -- 172.21.15.64:0/1085695142 wait complete. 2023-11-24T06:27:50.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.093+0000 7fa666683640 1 -- 172.21.15.64:0/1563269635 shutdown_connections 2023-11-24T06:27:50.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.093+0000 7fa666683640 1 -- 172.21.15.64:0/1563269635 wait complete. 2023-11-24T06:27:50.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.096+0000 7ff3196ff640 1 Processor -- start 2023-11-24T06:27:50.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.096+0000 7fa666683640 1 Processor -- start 2023-11-24T06:27:50.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.096+0000 7fa666683640 1 -- start start 2023-11-24T06:27:50.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.096+0000 7f3a67090640 1 Processor -- start 2023-11-24T06:27:50.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.097+0000 7f3a67090640 1 -- start start 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.096+0000 7ff3196ff640 1 -- start start 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff3196ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30c177320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff3196ff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c052030 0x7ff30c03e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff3196ff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff30c0524b0 0x7ff30c03e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff3196ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c177860 con 0x7ff304005180 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff3196ff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1809a0 con 0x7ff30c052030 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff3196ff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c180c80 con 0x7ff30c0524b0 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff313fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30c177320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff313fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30c177320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48478/0 (socket says 172.21.15.64:48478) 2023-11-24T06:27:50.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.091+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b40067450 con 0x7f4b40161aa0 2023-11-24T06:27:50.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.092+0000 7f4b35ffb640 1 -- 172.21.15.64:0/1361421223 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b380050d0 con 0x7f4b40161aa0 2023-11-24T06:27:50.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.094+0000 7f4b35ffb640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b2406ae40 0x7f4b2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.094+0000 7f4b35ffb640 1 -- 172.21.15.64:0/1361421223 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b38088a50 con 0x7f4b40161aa0 2023-11-24T06:27:50.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.097+0000 7f4b456b9640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b2406ae40 0x7f4b2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.097+0000 7f4b456b9640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b2406ae40 0x7f4b2406d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0094a0 tx=0x7f4b3c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7fa666683640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa66006e7e0 0x7fa6601f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7fa666683640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b3c10 0x7fa6600b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7fa666683640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b4280 0x7fa6600b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7f3a67090640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a600ae5a0 0x7f3a600b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7f3a67090640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b3ae0 0x7f3a6014cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.096+0000 7f2e6d7ff640 1 -- start start 2023-11-24T06:27:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7f2e6d7ff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e68160920 0x7f2e680b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7f2e6d7ff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e680d0a10 con 0x7f2e680cfea0 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7f2e6d7ff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e68161e30 con 0x7f2e680cf9f0 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7f2e6d7ff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e680d1f50 con 0x7f2e68160920 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7f2e67fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e68160920 0x7f2e680b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7f2e67fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e68160920 0x7f2e680b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39628/0 (socket says 172.21.15.64:39628) 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7f2e67fff640 1 -- 172.21.15.64:0/1295031844 learned_addr learned my addr 172.21.15.64:0/1295031844 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7f2e67fff640 1 -- 172.21.15.64:0/1295031844 >> v1:172.21.15.162:6789/0 conn(0x7f2e680cf9f0 legacy=0x7f2e68161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.100+0000 7f2e67fff640 1 -- 172.21.15.64:0/1295031844 >> v1:172.21.15.64:6789/0 conn(0x7f2e680cfea0 legacy=0x7f2e68161720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:27:50.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.100+0000 7f2e67fff640 1 -- 172.21.15.64:0/1295031844 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e680cf080 con 0x7f2e68160920 2023-11-24T06:27:50.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.098+0000 7ff313fff640 1 -- 172.21.15.64:0/1886108499 learned_addr learned my addr 172.21.15.64:0/1886108499 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7ff313fff640 1 -- 172.21.15.64:0/1886108499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff30c0524b0 msgr2=0x7ff30c03e880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:27:50.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7ff313fff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff30c0524b0 0x7ff30c03e880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7ff313fff640 1 -- 172.21.15.64:0/1886108499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c052030 msgr2=0x7ff30c03e340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:27:50.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7ff313fff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c052030 0x7ff30c03e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7ff313fff640 1 -- 172.21.15.64:0/1886108499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c180f60 con 0x7ff304005180 2023-11-24T06:27:50.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7ff313fff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30c177320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff30800d800 tx=0x7ff30800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7f3a67090640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a600cf290 0x7f3a60152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.100+0000 7f3a64e2c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a600ae5a0 0x7f3a600b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.100+0000 7f3a64e2c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a600ae5a0 0x7f3a600b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:33020/0 (socket says 172.21.15.64:33020) 2023-11-24T06:27:50.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.100+0000 7f3a64e2c640 1 -- 172.21.15.64:0/2191376345 learned_addr learned my addr 172.21.15.64:0/2191376345 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6014fdd0 con 0x7f3a600b3ae0 2023-11-24T06:27:50.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60151ab0 con 0x7f3a600cf290 2023-11-24T06:27:50.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600b0960 con 0x7f3a600ae5a0 2023-11-24T06:27:50.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.099+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa66006e7e0 0x7fa6601f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa66006e7e0 0x7fa6601f5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39640/0 (socket says 172.21.15.64:39640) 2023-11-24T06:27:50.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7fa65ffff640 1 -- 172.21.15.64:0/1022807110 learned_addr learned my addr 172.21.15.64:0/1022807110 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.100+0000 7fa6577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b3c10 0x7fa6600b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6601f0a40 con 0x7fa6600b3c10 2023-11-24T06:27:50.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6601f2a50 con 0x7fa66006e7e0 2023-11-24T06:27:50.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa660159540 con 0x7fa6600b4280 2023-11-24T06:27:50.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7f3a6562d640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a600cf290 0x7f3a60152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7fa664c20640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b4280 0x7fa6600b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.101+0000 7f3a5ffff640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b3ae0 0x7f3a6014cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7fa65ffff640 1 -- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b4280 msgr2=0x7fa6600b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7fa65ffff640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b4280 0x7fa6600b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7fa65ffff640 1 -- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b3c10 msgr2=0x7fa6600b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7f3a64e2c640 1 -- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a600cf290 msgr2=0x7f3a60152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7f3a64e2c640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a600cf290 0x7f3a60152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7f3a64e2c640 1 -- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b3ae0 msgr2=0x7f3a6014cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7f3a64e2c640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b3ae0 0x7f3a6014cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.102+0000 7f3a64e2c640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a540094d0 con 0x7f3a600ae5a0 2023-11-24T06:27:50.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7f3a5ffff640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b3ae0 0x7f3a6014cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:50.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7ff310ff9640 1 -- 172.21.15.64:0/1886108499 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080096c0 con 0x7ff304005180 2023-11-24T06:27:50.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7ff310ff9640 1 -- 172.21.15.64:0/1886108499 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308009820 con 0x7ff304005180 2023-11-24T06:27:50.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7ff310ff9640 1 -- 172.21.15.64:0/1886108499 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080096c0 con 0x7ff304005180 2023-11-24T06:27:50.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c181240 con 0x7ff304005180 2023-11-24T06:27:50.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7f4b35ffb640 1 -- 172.21.15.64:0/1361421223 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b38053bc0 con 0x7f4b40161aa0 2023-11-24T06:27:50.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7fa65ffff640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b3c10 0x7fa6600b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7fa65ffff640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650009b30 con 0x7fa66006e7e0 2023-11-24T06:27:50.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7fa6577fe640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b3c10 0x7fa6600b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:50.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7f2e67fff640 1 --2- 172.21.15.64:0/1295031844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e68160920 0x7f2e680b3f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009870 tx=0x7f2e5002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=191f7406db777a72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f3a64e2c640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a600ae5a0 0x7f3a600b0420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c7a0 tx=0x7f3a5800cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f3a5d7fa640 1 -- 172.21.15.64:0/2191376345 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58003d80 con 0x7f3a600ae5a0 2023-11-24T06:27:50.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f2e677fe640 1 -- 172.21.15.64:0/1295031844 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50030860 con 0x7f2e68160920 2023-11-24T06:27:50.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f2e677fe640 1 -- 172.21.15.64:0/1295031844 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e500309c0 con 0x7f2e68160920 2023-11-24T06:27:50.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f2e677fe640 1 -- 172.21.15.64:0/1295031844 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50030d50 con 0x7f2e68160920 2023-11-24T06:27:50.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a600afcd0 con 0x7f3a600ae5a0 2023-11-24T06:27:50.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f3a5d7fa640 1 -- 172.21.15.64:0/2191376345 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58005070 con 0x7f3a600ae5a0 2023-11-24T06:27:50.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7f3a5d7fa640 1 -- 172.21.15.64:0/2191376345 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58005460 con 0x7f3a600ae5a0 2023-11-24T06:27:50.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.104+0000 7fa65ffff640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa66006e7e0 0x7fa6601f5dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa650009e20 tx=0x7fa650004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.105+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/1295031844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e68160920 msgr2=0x7f2e680b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.105+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a60152030 con 0x7f3a600ae5a0 2023-11-24T06:27:50.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.105+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/1295031844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e68160920 0x7f2e680b3f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009870 tx=0x7f2e5002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.105+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/1295031844 shutdown_connections 2023-11-24T06:27:50.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.105+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/1295031844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e68160920 0x7f2e680b3f30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.105+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/1295031844 >> 172.21.15.64:0/1295031844 conn(0x7f2e680d3090 msgr2=0x7f2e680d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.105+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/1295031844 shutdown_connections 2023-11-24T06:27:50.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.106+0000 7fa65d7fa640 1 -- 172.21.15.64:0/1022807110 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65003e450 con 0x7fa66006e7e0 2023-11-24T06:27:50.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.106+0000 7fa65d7fa640 1 -- 172.21.15.64:0/1022807110 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65003e5b0 con 0x7fa66006e7e0 2023-11-24T06:27:50.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.106+0000 7fa65d7fa640 1 -- 172.21.15.64:0/1022807110 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65003e960 con 0x7fa66006e7e0 2023-11-24T06:27:50.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.106+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa660159820 con 0x7fa66006e7e0 2023-11-24T06:27:50.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.107+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa660159de0 con 0x7fa66006e7e0 2023-11-24T06:27:50.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.107+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/1295031844 wait complete. 2023-11-24T06:27:50.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.103+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c181890 con 0x7ff304005180 2023-11-24T06:27:50.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.108+0000 7ff310ff9640 1 -- 172.21.15.64:0/1886108499 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff308009990 con 0x7ff304005180 2023-11-24T06:27:50.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.110+0000 7ff310ff9640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff2f806aea0 0x7ff2f806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.111+0000 7ff310ff9640 1 -- 172.21.15.64:0/1886108499 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff308058070 con 0x7ff304005180 2023-11-24T06:27:50.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.109+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6600675d0 con 0x7fa66006e7e0 2023-11-24T06:27:50.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.109+0000 7f3a5d7fa640 1 -- 172.21.15.64:0/2191376345 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a580055c0 con 0x7f3a600ae5a0 2023-11-24T06:27:50.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.109+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60151910 con 0x7f3a600ae5a0 2023-11-24T06:27:50.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.110+0000 7f3a5d7fa640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3a4c06ae40 0x7f3a4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.111+0000 7f3a5d7fa640 1 -- 172.21.15.64:0/2191376345 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a58088ae0 con 0x7f3a600ae5a0 2023-11-24T06:27:50.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.111+0000 7f3a5ffff640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3a4c06ae40 0x7f3a4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.110+0000 7f2e6d7ff640 1 Processor -- start 2023-11-24T06:27:50.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.112+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c17d360 con 0x7ff304005180 2023-11-24T06:27:50.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.112+0000 7fa65d7fa640 1 -- 172.21.15.64:0/1022807110 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa65002cd50 con 0x7fa66006e7e0 2023-11-24T06:27:50.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.112+0000 7f3a5ffff640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3a4c06ae40 0x7f3a4c06d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a6015a3f0 tx=0x7f3a50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.113+0000 7ff3137fe640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff2f806aea0 0x7ff2f806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.113+0000 7f2e6d7ff640 1 -- start start 2023-11-24T06:27:50.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.114+0000 7fa65d7fa640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa64006ae40 0x7fa64006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.114+0000 7fa65d7fa640 1 -- 172.21.15.64:0/1022807110 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6500ac020 con 0x7fa66006e7e0 2023-11-24T06:27:50.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.115+0000 7ff3137fe640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff2f806aea0 0x7ff2f806d360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff3000079e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.115+0000 7fa6577fe640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa64006ae40 0x7fa64006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.116+0000 7f2e6d7ff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e680cf9f0 0x7f2e6815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.116+0000 7f2e6d7ff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e680cfea0 0x7f2e68157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.116+0000 7f2e6d7ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e68160920 0x7f2e68141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.116+0000 7fa6577fe640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa64006ae40 0x7fa64006d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa648006fd0 tx=0x7fa648008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.116+0000 7f2e6cffe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e68160920 0x7f2e68141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.116+0000 7f2e6cffe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e68160920 0x7f2e68141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48498/0 (socket says 172.21.15.64:48498) 2023-11-24T06:27:50.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.117+0000 7f2e6cffe640 1 -- 172.21.15.64:0/487988528 learned_addr learned my addr 172.21.15.64:0/487988528 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:50.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.117+0000 7f2e67fff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e680cf9f0 0x7f2e6815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.117+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6815a020 con 0x7f2e68160920 2023-11-24T06:27:50.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.117+0000 7f2e5f7fe640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e680cfea0 0x7f2e68157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.117+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e68159410 con 0x7f2e680cfea0 2023-11-24T06:27:50.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.117+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e681596f0 con 0x7f2e680cf9f0 2023-11-24T06:27:50.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.118+0000 7ff310ff9640 1 -- 172.21.15.64:0/1886108499 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff30808e060 con 0x7ff304005180 2023-11-24T06:27:50.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.118+0000 7f2e6cffe640 1 -- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e680cf9f0 msgr2=0x7f2e6815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.118+0000 7f2e6cffe640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e680cf9f0 0x7f2e6815a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.118+0000 7f2e6cffe640 1 -- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e680cfea0 msgr2=0x7f2e68157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.118+0000 7f2e6cffe640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e680cfea0 0x7f2e68157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.118+0000 7f2e6cffe640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e500094d0 con 0x7f2e68160920 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.119+0000 7f2e6cffe640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e68160920 0x7f2e68141e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000b3f0 tx=0x7f2e6000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.119+0000 7f2e657fa640 1 -- 172.21.15.64:0/487988528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60011830 con 0x7f2e68160920 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.119+0000 7f2e657fa640 1 -- 172.21.15.64:0/487988528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e60011990 con 0x7f2e68160920 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.119+0000 7f2e657fa640 1 -- 172.21.15.64:0/487988528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60011d40 con 0x7f2e68160920 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.119+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e681599d0 con 0x7f2e68160920 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.119+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e68158280 con 0x7f2e68160920 2023-11-24T06:27:50.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.121+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6815a190 con 0x7f2e68160920 2023-11-24T06:27:50.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.124+0000 7f2e657fa640 1 -- 172.21.15.64:0/487988528 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e6001a400 con 0x7f2e68160920 2023-11-24T06:27:50.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.125+0000 7f2e657fa640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2e4806ae40 0x7f2e4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.126+0000 7f2e657fa640 1 -- 172.21.15.64:0/487988528 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e600896c0 con 0x7f2e68160920 2023-11-24T06:27:50.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.126+0000 7f2e67fff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2e4806ae40 0x7f2e4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.127+0000 7fa65d7fa640 1 -- 172.21.15.64:0/1022807110 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6500dee90 con 0x7fa66006e7e0 2023-11-24T06:27:50.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.127+0000 7f2e67fff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2e4806ae40 0x7f2e4806d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e50036000 tx=0x7f2e50008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.128+0000 7f3a5d7fa640 1 -- 172.21.15.64:0/2191376345 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a60151910 con 0x7f3a600ae5a0 2023-11-24T06:27:50.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.133+0000 7f2e657fa640 1 -- 172.21.15.64:0/487988528 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e60054780 con 0x7f2e68160920 2023-11-24T06:27:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.497+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fab940057d0 con 0x7fabb40cfad0 2023-11-24T06:27:50.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.498+0000 7fab9bfff640 1 -- 172.21.15.64:0/223578030 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fabac0594a0 con 0x7fabb40cfad0 2023-11-24T06:27:50.502 INFO:teuthology.orchestra.run.smithi064.stdout:60129542147 2023-11-24T06:27:50.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.501+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7faba006af10 msgr2=0x7faba006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.502+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7faba006af10 0x7faba006d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabb414fd30 tx=0x7faba4009040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.502+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 msgr2=0x7fabb4150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.502+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4150bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d080 tx=0x7fabac00d540 comp rx=0 tx=0).stop 2023-11-24T06:27:50.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.502+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 shutdown_connections 2023-11-24T06:27:50.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.502+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7faba006af10 0x7faba006d3d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.503+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb41598d0 0x7fabb41f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.503+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4153a50 0x7fabb41f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.503+0000 7fabb9e97640 1 --2- 172.21.15.64:0/223578030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabb40cfad0 0x7fabb4150bd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.503+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 >> 172.21.15.64:0/223578030 conn(0x7fabb406a800 msgr2=0x7fabb4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.503+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 shutdown_connections 2023-11-24T06:27:50.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.503+0000 7fabb9e97640 1 -- 172.21.15.64:0/223578030 wait complete. 2023-11-24T06:27:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.508+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8fa80cd250 con 0x7f8fa8157170 2023-11-24T06:27:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.509+0000 7f8f9effd640 1 -- 172.21.15.64:0/144348322 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8fa001a090 con 0x7f8fa8157170 2023-11-24T06:27:50.516 INFO:teuthology.orchestra.run.smithi064.stdout:60129542148 2023-11-24T06:27:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.513+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8f8806aed0 msgr2=0x7f8f8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.513+0000 7f8fa759e640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8f8806aed0 0x7f8f8806d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f9002b980 tx=0x7f8f90008030 comp rx=0 tx=0).stop 2023-11-24T06:27:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.513+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8157170 msgr2=0x7f8fa8153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.513+0000 7f8fa759e640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8157170 0x7f8fa8153810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007f20 tx=0x7f8fa000d560 comp rx=0 tx=0).stop 2023-11-24T06:27:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.514+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 shutdown_connections 2023-11-24T06:27:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.514+0000 7f8fa759e640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8f8806aed0 0x7f8f8806d390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.514+0000 7f8fa759e640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8157170 0x7f8fa8153810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.514+0000 7f8fa759e640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa8162430 0x7f8fa8156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.514+0000 7f8fa759e640 1 --2- 172.21.15.64:0/144348322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fa80b3650 0x7f8fa81ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.514+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 >> 172.21.15.64:0/144348322 conn(0x7f8fa806a800 msgr2=0x7f8fa80b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.515+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 shutdown_connections 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.515+0000 7f8fa759e640 1 -- 172.21.15.64:0/144348322 wait complete. 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.519+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa7ac0cb590 con 0x7fa7ac0ce740 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.520+0000 7fa7a2ffd640 1 -- 172.21.15.64:0/4227926759 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa79c038020 con 0x7fa7ac0ce740 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi064.stdout:60129542148 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.524+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa79006af10 msgr2=0x7fa79006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.524+0000 7fa7b4432640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa79006af10 0x7fa79006d3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac15cc50 tx=0x7fa7a8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.524+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 msgr2=0x7fa7ac097cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.524+0000 7fa7b4432640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac097cc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa79c033000 tx=0x7fa79c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.524+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 shutdown_connections 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.524+0000 7fa7b4432640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa79006af10 0x7fa79006d3d0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac15cc50 tx=0x7fa7a8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.524+0000 7fa7b4432640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7ac142c30 0x7fa7ac06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.525+0000 7fa7b4432640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac1426e0 0x7fa7ac06cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.525+0000 7fa7b4432640 1 --2- 172.21.15.64:0/4227926759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7ac0ce740 0x7fa7ac097cc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.525+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 >> 172.21.15.64:0/4227926759 conn(0x7fa7ac0b4710 msgr2=0x7fa7ac0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.526+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 shutdown_connections 2023-11-24T06:27:50.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.527+0000 7fa7b4432640 1 -- 172.21.15.64:0/4227926759 wait complete. 2023-11-24T06:27:50.536 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.0 2023-11-24T06:27:50.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.535+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff30c005360 con 0x7ff304005180 2023-11-24T06:27:50.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.536+0000 7ff310ff9640 1 -- 172.21.15.64:0/1886108499 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff308053cb0 con 0x7ff304005180 2023-11-24T06:27:50.540 INFO:teuthology.orchestra.run.smithi064.stdout:60129542148 2023-11-24T06:27:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.538+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff2f806aea0 msgr2=0x7ff2f806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.538+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff2f806aea0 0x7ff2f806d360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff3000079e0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.538+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 msgr2=0x7ff30c177320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.538+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30c177320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff30800d800 tx=0x7ff30800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.538+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4b400cd800 con 0x7f4b40161aa0 2023-11-24T06:27:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.539+0000 7f4b35ffb640 1 -- 172.21.15.64:0/1361421223 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b380571e0 con 0x7f4b40161aa0 2023-11-24T06:27:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.539+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 shutdown_connections 2023-11-24T06:27:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.539+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff2f806aea0 0x7ff2f806d360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.539+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff30c0524b0 0x7ff30c03e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.539+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c052030 0x7ff30c03e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.539+0000 7ff3196ff640 1 --2- 172.21.15.64:0/1886108499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff304005180 0x7ff30c177320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.539+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 >> 172.21.15.64:0/1886108499 conn(0x7ff30c058eb0 msgr2=0x7ff30c037e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.544 INFO:teuthology.orchestra.run.smithi064.stdout:60129542148 2023-11-24T06:27:50.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.2 2023-11-24T06:27:50.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.542+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 shutdown_connections 2023-11-24T06:27:50.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.542+0000 7ff3196ff640 1 -- 172.21.15.64:0/1886108499 wait complete. 2023-11-24T06:27:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.542+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b2406ae40 msgr2=0x7f4b2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.542+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b2406ae40 0x7f4b2406d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0094a0 tx=0x7f4b3c009240 comp rx=0 tx=0).stop 2023-11-24T06:27:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.542+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b40161aa0 msgr2=0x7f4b4014a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.542+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b40161aa0 0x7f4b4014a930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800c730 tx=0x7f4b3800cbf0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 shutdown_connections 2023-11-24T06:27:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b2406ae40 0x7f4b2406d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b40161aa0 0x7f4b4014a930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b4015bd40 0x7f4b400c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 --2- 172.21.15.64:0/1361421223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b4006d0f0 0x7f4b4015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 >> 172.21.15.64:0/1361421223 conn(0x7f4b400b0b70 msgr2=0x7f4b400b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 shutdown_connections 2023-11-24T06:27:50.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.543+0000 7f4b466bb640 1 -- 172.21.15.64:0/1361421223 wait complete. 2023-11-24T06:27:50.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.1 2023-11-24T06:27:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.555+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3a60151910 con 0x7f3a600ae5a0 2023-11-24T06:27:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.556+0000 7f3a5d7fa640 1 -- 172.21.15.64:0/2191376345 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a60151910 con 0x7f3a600ae5a0 2023-11-24T06:27:50.560 INFO:teuthology.orchestra.run.smithi064.stdout:60129542148 2023-11-24T06:27:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3a4c06ae40 msgr2=0x7f3a4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3a4c06ae40 0x7f3a4c06d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a6015a3f0 tx=0x7f3a50008040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a600ae5a0 msgr2=0x7f3a600b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a600ae5a0 0x7f3a600b0420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c7a0 tx=0x7f3a5800cc60 comp rx=0 tx=0).stop 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 shutdown_connections 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3a4c06ae40 0x7f3a4c06d300 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a6015a3f0 tx=0x7f3a50008040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a600cf290 0x7f3a60152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b3ae0 0x7f3a6014cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 --2- 172.21.15.64:0/2191376345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a600ae5a0 0x7f3a600b0420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.559+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 >> 172.21.15.64:0/2191376345 conn(0x7f3a6006a800 msgr2=0x7f3a600b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.6 2023-11-24T06:27:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.562+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 shutdown_connections 2023-11-24T06:27:50.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.562+0000 7f3a67090640 1 -- 172.21.15.64:0/2191376345 wait complete. 2023-11-24T06:27:50.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.3 2023-11-24T06:27:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.563+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa660159f70 con 0x7fa66006e7e0 2023-11-24T06:27:50.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.564+0000 7fa65d7fa640 1 -- 172.21.15.64:0/1022807110 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa650077170 con 0x7fa66006e7e0 2023-11-24T06:27:50.573 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-11-24T06:27:50.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.565+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2e68067450 con 0x7f2e68160920 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa64006ae40 msgr2=0x7fa64006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa64006ae40 0x7fa64006d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa648006fd0 tx=0x7fa648008040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa66006e7e0 msgr2=0x7fa6601f5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa66006e7e0 0x7fa6601f5dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa650009e20 tx=0x7fa650004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 shutdown_connections 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa64006ae40 0x7fa64006d300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6600b4280 0x7fa6600b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6600b3c10 0x7fa6600b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 --2- 172.21.15.64:0/1022807110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa66006e7e0 0x7fa6601f5dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 >> 172.21.15.64:0/1022807110 conn(0x7fa6600cfef0 msgr2=0x7fa6600afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7f2e657fa640 1 -- 172.21.15.64:0/487988528 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e60057da0 con 0x7f2e68160920 2023-11-24T06:27:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 shutdown_connections 2023-11-24T06:27:50.575 INFO:teuthology.orchestra.run.smithi064.stdout:60129542147 2023-11-24T06:27:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.566+0000 7fa666683640 1 -- 172.21.15.64:0/1022807110 wait complete. 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2e4806ae40 msgr2=0x7f2e4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2e4806ae40 0x7f2e4806d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e50036000 tx=0x7f2e50008030 comp rx=0 tx=0).stop 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e68160920 msgr2=0x7f2e68141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e68160920 0x7f2e68141e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000b3f0 tx=0x7f2e6000b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 shutdown_connections 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2e4806ae40 0x7f2e4806d300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e68160920 0x7f2e68141e60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e680cfea0 0x7f2e68157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 --2- 172.21.15.64:0/487988528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e680cf9f0 0x7f2e6815a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 >> 172.21.15.64:0/487988528 conn(0x7f2e680d3090 msgr2=0x7f2e680c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 shutdown_connections 2023-11-24T06:27:50.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:50.569+0000 7f2e6d7ff640 1 -- 172.21.15.64:0/487988528 wait complete. 2023-11-24T06:27:50.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.7 2023-11-24T06:27:50.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.4 2023-11-24T06:27:50.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.5 2023-11-24T06:27:51.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:27:51.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:27:51.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:27:51.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:27:51.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:27:51.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:27:51.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:27:51.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:27:51.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.846+0000 7f6c5f0e7640 1 Processor -- start 2023-11-24T06:27:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.846+0000 7f6c5f0e7640 1 -- start start 2023-11-24T06:27:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.848+0000 7f6c5f0e7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c5815d560 0x7f6c580d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.848+0000 7f6c5f0e7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6c580b22b0 0x7f6c580cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.848+0000 7f6c5f0e7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.848+0000 7f6c5f0e7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580d03c0 con 0x7f6c5815d560 2023-11-24T06:27:51.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.848+0000 7f6c5f0e7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580d1970 con 0x7f6c580b22b0 2023-11-24T06:27:51.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.848+0000 7f6c5f0e7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580d1c50 con 0x7f6c580cff70 2023-11-24T06:27:51.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.850+0000 7f6c5d684640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.850+0000 7f6c5d684640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:33058/0 (socket says 172.21.15.64:33058) 2023-11-24T06:27:51.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.850+0000 7f6c5ce83640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c5815d560 0x7f6c580d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.851+0000 7f6c5ce83640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c5815d560 0x7f6c580d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48512/0 (socket says 172.21.15.64:48512) 2023-11-24T06:27:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.851+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1048675879 learned_addr learned my addr 172.21.15.64:0/1048675879 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.851+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1048675879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 msgr2=0x7f6c580cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.850+0000 7f6c57fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6c580b22b0 0x7f6c580cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.852+0000 7f6c5ce83640 1 --2- 172.21.15.64:0/1048675879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.852+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1048675879 >> v2:172.21.15.162:3300/0 conn(0x7f6c580b22b0 msgr2=0x7f6c580cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.852+0000 7f6c5ce83640 1 --2- 172.21.15.64:0/1048675879 >> v2:172.21.15.162:3300/0 conn(0x7f6c580b22b0 0x7f6c580cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.852+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1048675879 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580d1f30 con 0x7f6c5815d560 2023-11-24T06:27:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.852+0000 7f6c5d684640 1 --2- 172.21.15.64:0/1048675879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-11-24T06:27:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.852+0000 7f6c57fff640 1 --2- 172.21.15.64:0/1048675879 >> v2:172.21.15.162:3300/0 conn(0x7f6c580b22b0 0x7f6c580cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:51.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.854+0000 7f6c5ce83640 1 --2- 172.21.15.64:0/1048675879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580d0f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c5808d550 tx=0x7f6c4c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38754bf057ef124 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.855+0000 7f6c577fe640 1 -- 172.21.15.64:0/1048675879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c031870 con 0x7f6c5815d560 2023-11-24T06:27:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.856+0000 7f6c577fe640 1 -- 172.21.15.64:0/1048675879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c0319d0 con 0x7f6c5815d560 2023-11-24T06:27:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.856+0000 7f6c577fe640 1 -- 172.21.15.64:0/1048675879 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c031d60 con 0x7f6c5815d560 2023-11-24T06:27:51.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.857+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1048675879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 msgr2=0x7f6c580d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.857+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1048675879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580d0f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c5808d550 tx=0x7f6c4c02b910 comp rx=0 tx=0).stop 2023-11-24T06:27:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.858+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1048675879 shutdown_connections 2023-11-24T06:27:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.858+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1048675879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.858+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1048675879 >> v2:172.21.15.162:3300/0 conn(0x7f6c580b22b0 0x7f6c580cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.858+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1048675879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580d0f20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.858+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1048675879 >> 172.21.15.64:0/1048675879 conn(0x7f6c580d3020 msgr2=0x7f6c580d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.859+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1048675879 shutdown_connections 2023-11-24T06:27:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.859+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1048675879 wait complete. 2023-11-24T06:27:51.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.863+0000 7f6c5f0e7640 1 Processor -- start 2023-11-24T06:27:51.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.864+0000 7f6c5f0e7640 1 -- start start 2023-11-24T06:27:51.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.866+0000 7f6c5f0e7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c580b22b0 0x7f6c580c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.866+0000 7f6c5f0e7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.866+0000 7f6c5f0e7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.866+0000 7f6c5f0e7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b6800 con 0x7f6c5815d560 2023-11-24T06:27:51.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.866+0000 7f6c5f0e7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b15c0 con 0x7f6c580b22b0 2023-11-24T06:27:51.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.866+0000 7f6c5f0e7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580c61b0 con 0x7f6c580cff70 2023-11-24T06:27:51.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.867+0000 7f6c5d684640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.867+0000 7f6c5ce83640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c580b22b0 0x7f6c580c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.868+0000 7f6c5ce83640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c580b22b0 0x7f6c580c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39682/0 (socket says 172.21.15.64:39682) 2023-11-24T06:27:51.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.868+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1303706169 learned_addr learned my addr 172.21.15.64:0/1303706169 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.869+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1303706169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 msgr2=0x7f6c580b5c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:27:51.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.869+0000 7f6c5ce83640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580b5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.869+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 msgr2=0x7f6c580b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.869+0000 7f6c5ce83640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.869+0000 7f6c5ce83640 1 -- 172.21.15.64:0/1303706169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c0094d0 con 0x7f6c580b22b0 2023-11-24T06:27:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.869+0000 7f6c5d684640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.870+0000 7f6c5ce83640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c580b22b0 0x7f6c580c5c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0090b0 tx=0x7f6c4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.871+0000 7f6c557fa640 1 -- 172.21.15.64:0/1303706169 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c0443f0 con 0x7f6c580b22b0 2023-11-24T06:27:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.871+0000 7f6c557fa640 1 -- 172.21.15.64:0/1303706169 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c044550 con 0x7f6c580b22b0 2023-11-24T06:27:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.871+0000 7f6c557fa640 1 -- 172.21.15.64:0/1303706169 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c0443f0 con 0x7f6c580b22b0 2023-11-24T06:27:51.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.871+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c580b09b0 con 0x7f6c580b22b0 2023-11-24T06:27:51.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.872+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c580b0f70 con 0x7f6c580b22b0 2023-11-24T06:27:51.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.874+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c58067450 con 0x7f6c580b22b0 2023-11-24T06:27:51.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.875+0000 7f6c557fa640 1 -- 172.21.15.64:0/1303706169 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c4c0446c0 con 0x7f6c580b22b0 2023-11-24T06:27:51.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.877+0000 7f6c557fa640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6c4406aea0 0x7f6c4406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.877+0000 7f6c557fa640 1 -- 172.21.15.64:0/1303706169 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c4c02d040 con 0x7f6c580b22b0 2023-11-24T06:27:51.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.878+0000 7f6c57fff640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6c4406aea0 0x7f6c4406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.879+0000 7f6c57fff640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6c4406aea0 0x7f6c4406d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c48006b10 tx=0x7f6c48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.891+0000 7f94a4568640 1 Processor -- start 2023-11-24T06:27:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.891+0000 7f94a4568640 1 -- start start 2023-11-24T06:27:51.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.893+0000 7f6c557fa640 1 -- 172.21.15.64:0/1303706169 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c4c076f40 con 0x7f6c580b22b0 2023-11-24T06:27:51.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.893+0000 7fe47b807640 1 Processor -- start 2023-11-24T06:27:51.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.894+0000 7fe47b807640 1 -- start start 2023-11-24T06:27:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a4568640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a4568640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f949c0ce6f0 0x7f949c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a2304640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f949c15d590 0x7f949c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:53982/0 (socket says 172.21.15.64:53982) 2023-11-24T06:27:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a4568640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c0b6750 con 0x7f949c15d590 2023-11-24T06:27:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a2304640 1 -- 172.21.15.64:0/101184486 learned_addr learned my addr 172.21.15.64:0/101184486 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a4568640 1 -- 172.21.15.64:0/101184486 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c0cf1f0 con 0x7f949c0ce6f0 2023-11-24T06:27:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a4568640 1 -- 172.21.15.64:0/101184486 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0cef10 con 0x7f949c0d0280 2023-11-24T06:27:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7fe47b807640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe4740ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7fe47b807640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4740b30b0 0x7fe47406d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7fe47b807640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47415d840 con 0x7fe47406e380 2023-11-24T06:27:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7fe47b807640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe474071490 con 0x7fe4740b30b0 2023-11-24T06:27:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7fe47b807640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b3790 con 0x7fe47415d460 2023-11-24T06:27:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.895+0000 7f94a1b03640 1 --2- 172.21.15.64:0/101184486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.896+0000 7f94a2b05640 1 --2- 172.21.15.64:0/101184486 >> v2:172.21.15.64:3300/0 conn(0x7f949c0ce6f0 0x7f949c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.897+0000 7fe479da4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4740b30b0 0x7fe47406d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.897+0000 7fe4795a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe4740ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.897+0000 7fe4795a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe4740ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:33084/0 (socket says 172.21.15.64:33084) 2023-11-24T06:27:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.897+0000 7fe479da4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4740b30b0 0x7fe47406d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48540/0 (socket says 172.21.15.64:48540) 2023-11-24T06:27:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.898+0000 7fe479da4640 1 -- 172.21.15.64:0/1512521993 learned_addr learned my addr 172.21.15.64:0/1512521993 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.896+0000 7f94a1302640 1 -- 172.21.15.64:0/101184486 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882853056 0 0) 0x7f949c0b6750 con 0x7f949c15d590 2023-11-24T06:27:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.896+0000 7f94a1302640 1 -- 172.21.15.64:0/101184486 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9488003520 con 0x7f949c15d590 2023-11-24T06:27:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.896+0000 7f94a2b05640 1 -- 172.21.15.64:0/101184486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 msgr2=0x7f949c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.897+0000 7fe478da2640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe47406e380 0x7fe4740b29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44844/0 (socket says 172.21.15.64:44844) 2023-11-24T06:27:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.896+0000 7f94a2b05640 1 --2- 172.21.15.64:0/101184486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.896+0000 7f94a2b05640 1 -- 172.21.15.64:0/101184486 >> v1:172.21.15.162:6789/0 conn(0x7f949c15d590 legacy=0x7f949c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.896+0000 7f94a2b05640 1 -- 172.21.15.64:0/101184486 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0b6100 con 0x7f949c0ce6f0 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.899+0000 7fe479da4640 1 -- 172.21.15.64:0/1512521993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 msgr2=0x7fe4740ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.899+0000 7fe479da4640 1 --2- 172.21.15.64:0/1512521993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe4740ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.899+0000 7f94a2b05640 1 --2- 172.21.15.64:0/101184486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c0b6be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9494009d90 tx=0x7f949402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42afd98ac3280c89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.900+0000 7f94a1302640 1 -- 172.21.15.64:0/101184486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949402cc90 con 0x7f949c0ce6f0 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.900+0000 7f94a1302640 1 -- 172.21.15.64:0/101184486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949402cdf0 con 0x7f949c0ce6f0 2023-11-24T06:27:51.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.900+0000 7f94a1302640 1 -- 172.21.15.64:0/101184486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494034660 con 0x7f949c0ce6f0 2023-11-24T06:27:51.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.899+0000 7fe479da4640 1 -- 172.21.15.64:0/1512521993 >> v1:172.21.15.64:6789/0 conn(0x7fe47406e380 legacy=0x7fe4740b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.899+0000 7fe473fff640 1 -- 172.21.15.64:0/1512521993 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633241903 0 0) 0x7fe47415d840 con 0x7fe47406e380 2023-11-24T06:27:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.899+0000 7fe479da4640 1 -- 172.21.15.64:0/1512521993 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474072190 con 0x7fe4740b30b0 2023-11-24T06:27:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.901+0000 7fe4795a3640 1 --2- 172.21.15.64:0/1512521993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe4740ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.901+0000 7f94a4568640 1 -- 172.21.15.64:0/101184486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 msgr2=0x7f949c0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.901+0000 7f94a4568640 1 --2- 172.21.15.64:0/101184486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c0b6be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9494009d90 tx=0x7f949402bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.901+0000 7fe479da4640 1 --2- 172.21.15.64:0/1512521993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4740b30b0 0x7fe47406d280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe464009870 tx=0x7fe46402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b7bd7d3b17134d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.902+0000 7f94a4568640 1 -- 172.21.15.64:0/101184486 shutdown_connections 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.902+0000 7f94a4568640 1 --2- 172.21.15.64:0/101184486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c0b6be0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.902+0000 7f94a4568640 1 --2- 172.21.15.64:0/101184486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.902+0000 7f94a4568640 1 -- 172.21.15.64:0/101184486 >> 172.21.15.64:0/101184486 conn(0x7f949c09c540 msgr2=0x7f949c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.902+0000 7f94a4568640 1 -- 172.21.15.64:0/101184486 shutdown_connections 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.902+0000 7f94a4568640 1 -- 172.21.15.64:0/101184486 wait complete. 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.903+0000 7fe473fff640 1 -- 172.21.15.64:0/1512521993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464030860 con 0x7fe4740b30b0 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.903+0000 7fe473fff640 1 -- 172.21.15.64:0/1512521993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4640309c0 con 0x7fe4740b30b0 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.903+0000 7fe473fff640 1 -- 172.21.15.64:0/1512521993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464030d50 con 0x7fe4740b30b0 2023-11-24T06:27:51.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.903+0000 7f0b66190640 1 Processor -- start 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.904+0000 7fe47b807640 1 -- 172.21.15.64:0/1512521993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4740b30b0 msgr2=0x7fe47406d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.904+0000 7fe47b807640 1 --2- 172.21.15.64:0/1512521993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4740b30b0 0x7fe47406d280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe464009870 tx=0x7fe46402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7fe47b807640 1 -- 172.21.15.64:0/1512521993 shutdown_connections 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7fe47b807640 1 --2- 172.21.15.64:0/1512521993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4740b30b0 0x7fe47406d280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7fe47b807640 1 --2- 172.21.15.64:0/1512521993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe4740ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7fe47b807640 1 -- 172.21.15.64:0/1512521993 >> 172.21.15.64:0/1512521993 conn(0x7fe4740b0be0 msgr2=0x7fe4740b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.903+0000 7f0b66190640 1 -- start start 2023-11-24T06:27:51.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7fe47b807640 1 -- 172.21.15.64:0/1512521993 shutdown_connections 2023-11-24T06:27:51.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7f94a4568640 1 Processor -- start 2023-11-24T06:27:51.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.906+0000 7f94a4568640 1 -- start start 2023-11-24T06:27:51.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7f0b66190640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b6006b690 0x7f0b601540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.906+0000 7f0b66190640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6015d550 con 0x7f0b6015dd30 2023-11-24T06:27:51.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.906+0000 7f0b66190640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6015a7b0 con 0x7f0b60159f60 2023-11-24T06:27:51.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.906+0000 7f0b66190640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b60066980 con 0x7f0b6006b690 2023-11-24T06:27:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.905+0000 7fe47b807640 1 -- 172.21.15.64:0/1512521993 wait complete. 2023-11-24T06:27:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.907+0000 7f0b5f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b6006b690 0x7f0b601540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.907+0000 7f0b5f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b6006b690 0x7f0b601540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48552/0 (socket says 172.21.15.64:48552) 2023-11-24T06:27:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.907+0000 7f0b5effd640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0b60159f60 0x7f0b60066270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:53998/0 (socket says 172.21.15.64:53998) 2023-11-24T06:27:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.907+0000 7f0b5effd640 1 -- 172.21.15.64:0/4255694160 learned_addr learned my addr 172.21.15.64:0/4255694160 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.908+0000 7f94a4568640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c1f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.908+0000 7f94a2304640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c1f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.909+0000 7f94a4568640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.909+0000 7f94a2304640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c1f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48554/0 (socket says 172.21.15.64:48554) 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.909+0000 7f94a2304640 1 -- 172.21.15.64:0/3460749134 learned_addr learned my addr 172.21.15.64:0/3460749134 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.908+0000 7f0b5f7fe640 1 -- 172.21.15.64:0/4255694160 >> v1:172.21.15.162:6789/0 conn(0x7f0b60159f60 legacy=0x7f0b60066270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.908+0000 7f0b5f7fe640 1 -- 172.21.15.64:0/4255694160 >> v1:172.21.15.64:6789/0 conn(0x7f0b6015dd30 legacy=0x7f0b60070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.908+0000 7f0b5f7fe640 1 -- 172.21.15.64:0/4255694160 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6015aa90 con 0x7f0b6006b690 2023-11-24T06:27:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.909+0000 7f94a4568640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f949c15d590 0x7f949c0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7fe47b807640 1 Processor -- start 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7f94a1b03640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7f0b5f7fe640 1 --2- 172.21.15.64:0/4255694160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6006b690 0x7f0b601540b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009d90 tx=0x7f0b5802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7283a0fe75ab7df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0cb200 con 0x7f949c0ce6f0 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0a0b30 con 0x7f949c15d590 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7fe47b807640 1 -- start start 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7f0b5e7fc640 1 -- 172.21.15.64:0/4255694160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5802cc60 con 0x7f0b6006b690 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.910+0000 7f94a2b05640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f949c15d590 0x7f949c0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7f0b5e7fc640 1 -- 172.21.15.64:0/4255694160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5802cdc0 con 0x7f0b6006b690 2023-11-24T06:27:51.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7f0b5e7fc640 1 -- 172.21.15.64:0/4255694160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58034660 con 0x7f0b6006b690 2023-11-24T06:27:51.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0cb450 con 0x7f949c0d0280 2023-11-24T06:27:51.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7fe47b807640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47406e380 0x7fe474152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7fe47b807640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4740b30b0 0x7fe47406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7fe47b807640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe47406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7f0b66190640 1 -- 172.21.15.64:0/4255694160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6006b690 msgr2=0x7f0b601540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7f0b66190640 1 --2- 172.21.15.64:0/4255694160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6006b690 0x7f0b601540b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009d90 tx=0x7f0b5802beb0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7f94a2304640 1 -- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 msgr2=0x7f949c0c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7f94a2304640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7f94a2304640 1 -- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f949c15d590 msgr2=0x7f949c0c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7f94a2304640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f949c15d590 0x7f949c0c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7fe47b807640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe474158780 con 0x7fe47406e380 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7fe47b807640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4741f7e80 con 0x7fe4740b30b0 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7fe47b807640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4741f72b0 con 0x7fe47415d460 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7fe478da2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4740b30b0 0x7fe47406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7fe478da2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4740b30b0 0x7fe47406d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39704/0 (socket says 172.21.15.64:39704) 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7fe478da2640 1 -- 172.21.15.64:0/722903450 learned_addr learned my addr 172.21.15.64:0/722903450 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7f0b66190640 1 -- 172.21.15.64:0/4255694160 shutdown_connections 2023-11-24T06:27:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7f0b66190640 1 --2- 172.21.15.64:0/4255694160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6006b690 0x7f0b601540b0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009d90 tx=0x7f0b5802beb0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.912+0000 7f0b66190640 1 -- 172.21.15.64:0/4255694160 >> 172.21.15.64:0/4255694160 conn(0x7f0b600cf700 msgr2=0x7f0b600cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7f94a2304640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494009a70 con 0x7f949c0ce6f0 2023-11-24T06:27:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7f94a1b03640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7fe479da4640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe47406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7fe4795a3640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47406e380 0x7fe474152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7f6ff5ab4640 1 Processor -- start 2023-11-24T06:27:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.911+0000 7f6ff5ab4640 1 -- start start 2023-11-24T06:27:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.913+0000 7fa4f113e640 1 Processor -- start 2023-11-24T06:27:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7fa4f113e640 1 -- start start 2023-11-24T06:27:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7fe478da2640 1 -- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 msgr2=0x7fe47406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7f94a2304640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c1f5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f949800c430 tx=0x7f949800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7f0b66190640 1 -- 172.21.15.64:0/4255694160 shutdown_connections 2023-11-24T06:27:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7fe478da2640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe47406d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7fe478da2640 1 -- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47406e380 msgr2=0x7fe474152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7f6ff5ab4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6ff00c6d80 0x7f6ff00d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7f6ff5ab4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff00cdd40 con 0x7f6ff00d1540 2023-11-24T06:27:51.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7f6ff5ab4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff00ce000 con 0x7f6ff00d1e60 2023-11-24T06:27:51.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7f6ff5ab4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff00d0ab0 con 0x7f6ff00c6d80 2023-11-24T06:27:51.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7fe478da2640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47406e380 0x7fe474152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7fe478da2640 1 -- 172.21.15.64:0/722903450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640094d0 con 0x7fe4740b30b0 2023-11-24T06:27:51.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7fe478da2640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4740b30b0 0x7fe47406d180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c7c0 tx=0x7fe46800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.914+0000 7f0b66190640 1 -- 172.21.15.64:0/4255694160 wait complete. 2023-11-24T06:27:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f9492ffd640 1 -- 172.21.15.64:0/3460749134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498005ce0 con 0x7f949c0ce6f0 2023-11-24T06:27:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f9492ffd640 1 -- 172.21.15.64:0/3460749134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498005e40 con 0x7f949c0ce6f0 2023-11-24T06:27:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f9492ffd640 1 -- 172.21.15.64:0/3460749134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94980072f0 con 0x7f949c0ce6f0 2023-11-24T06:27:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c0cb730 con 0x7f949c0ce6f0 2023-11-24T06:27:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f6fe7fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6ff00c6d80 0x7f6ff00d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f6fe7fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6ff00c6d80 0x7f6ff00d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39712/0 (socket says 172.21.15.64:39712) 2023-11-24T06:27:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f6fe7fff640 1 -- 172.21.15.64:0/1444490816 learned_addr learned my addr 172.21.15.64:0/1444490816 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c1f5db0 con 0x7f949c0ce6f0 2023-11-24T06:27:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.915+0000 7fe471ffb640 1 -- 172.21.15.64:0/722903450 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800a640 con 0x7fe4740b30b0 2023-11-24T06:27:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.916+0000 7fe471ffb640 1 -- 172.21.15.64:0/722903450 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46800a7a0 con 0x7fe4740b30b0 2023-11-24T06:27:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.916+0000 7fe471ffb640 1 -- 172.21.15.64:0/722903450 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800ab30 con 0x7fe4740b30b0 2023-11-24T06:27:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.916+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4741f7550 con 0x7fe4740b30b0 2023-11-24T06:27:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.916+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4741f7b10 con 0x7fe4740b30b0 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4f113e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4ec15d320 0x7fa4ec06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4f113e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ec153d30 con 0x7fa4ec15ce90 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4f113e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ec0b2b70 con 0x7fa4ec153950 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4f113e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ec0b2e50 con 0x7fa4ec15d320 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4ea59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa4ec15ce90 0x7fa4ec06c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44854/0 (socket says 172.21.15.64:44854) 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4ea59c640 1 -- 172.21.15.64:0/2635971430 learned_addr learned my addr 172.21.15.64:0/2635971430 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4ead9d640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa4ec153950 0x7fa4ec0cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:54024/0 (socket says 172.21.15.64:54024) 2023-11-24T06:27:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7fa4eb59e640 1 --2- 172.21.15.64:0/2635971430 >> v2:172.21.15.162:3300/0 conn(0x7fa4ec15d320 0x7fa4ec06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.918+0000 7f0b66190640 1 Processor -- start 2023-11-24T06:27:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.918+0000 7f0b66190640 1 -- start start 2023-11-24T06:27:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7f6fe7fff640 1 -- 172.21.15.64:0/1444490816 >> v1:172.21.15.162:6789/0 conn(0x7f6ff00d1e60 legacy=0x7f6ff00d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.917+0000 7f6feeffd640 1 -- 172.21.15.64:0/1444490816 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722709823 0 0) 0x7f6ff00ce000 con 0x7f6ff00d1e60 2023-11-24T06:27:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.918+0000 7f6fe7fff640 1 -- 172.21.15.64:0/1444490816 >> v1:172.21.15.64:6789/0 conn(0x7f6ff00d1540 legacy=0x7f6ff00cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.918+0000 7f6fe7fff640 1 -- 172.21.15.64:0/1444490816 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff00d0210 con 0x7f6ff00c6d80 2023-11-24T06:27:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.918+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4740710c0 con 0x7fe4740b30b0 2023-11-24T06:27:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.919+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9484005250 con 0x7f949c0ce6f0 2023-11-24T06:27:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.918+0000 7fa4e9d9b640 1 -- 172.21.15.64:0/2635971430 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756907894 0 0) 0x7fa4ec0b2b70 con 0x7fa4ec153950 2023-11-24T06:27:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.918+0000 7fa4e9d9b640 1 -- 172.21.15.64:0/2635971430 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d4003520 con 0x7fa4ec153950 2023-11-24T06:27:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.919+0000 7fa4eb59e640 1 -- 172.21.15.64:0/2635971430 >> v1:172.21.15.162:6789/0 conn(0x7fa4ec153950 legacy=0x7fa4ec0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.919+0000 7fa4eb59e640 1 -- 172.21.15.64:0/2635971430 >> v1:172.21.15.64:6789/0 conn(0x7fa4ec15ce90 legacy=0x7fa4ec06c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.919+0000 7fa4eb59e640 1 -- 172.21.15.64:0/2635971430 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec0b3130 con 0x7fa4ec15d320 2023-11-24T06:27:51.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.920+0000 7f0b66190640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b60159f60 0x7f0b60150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.921+0000 7f9492ffd640 1 -- 172.21.15.64:0/3460749134 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9498007450 con 0x7f949c0ce6f0 2023-11-24T06:27:51.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.921+0000 7f0b66190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6015dd30 0x7f0b60158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.921+0000 7fa4eb59e640 1 --2- 172.21.15.64:0/2635971430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec15d320 0x7fa4ec06cb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b6e2b114de5d93 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.921+0000 7f0b66190640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b601514e0 0x7f0b60159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.921+0000 7f0b66190640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6015b6a0 con 0x7f0b6015dd30 2023-11-24T06:27:51.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.921+0000 7f0b66190640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600ce5f0 con 0x7f0b60159f60 2023-11-24T06:27:51.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.921+0000 7f0b66190640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600ce310 con 0x7f0b601514e0 2023-11-24T06:27:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.922+0000 7f0b5f7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b60159f60 0x7f0b60150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.922+0000 7f0b5f7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b60159f60 0x7f0b60150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39734/0 (socket says 172.21.15.64:39734) 2023-11-24T06:27:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.922+0000 7f0b5f7fe640 1 -- 172.21.15.64:0/1805252355 learned_addr learned my addr 172.21.15.64:0/1805252355 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.922+0000 7f0b5ffff640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b601514e0 0x7f0b60159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.922+0000 7fe471ffb640 1 -- 172.21.15.64:0/722903450 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe46800ac90 con 0x7fe4740b30b0 2023-11-24T06:27:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.923+0000 7f0b5f7fe640 1 -- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b601514e0 msgr2=0x7f0b60159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.923+0000 7f0b5f7fe640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b601514e0 0x7f0b60159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.923+0000 7f0b5f7fe640 1 -- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6015dd30 msgr2=0x7f0b60158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.923+0000 7f0b5f7fe640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6015dd30 0x7f0b60158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.923+0000 7f0b5f7fe640 1 -- 172.21.15.64:0/1805252355 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58009a70 con 0x7f0b60159f60 2023-11-24T06:27:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.923+0000 7f0b5ffff640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b601514e0 0x7f0b60159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.923+0000 7f0b5f7fe640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b60159f60 0x7f0b60150fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b58037040 tx=0x7f0b58004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.922+0000 7f6fe7fff640 1 --2- 172.21.15.64:0/1444490816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00d1920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009870 tx=0x7f6fd802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=614888bdbcf22b4f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.924+0000 7f9492ffd640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f948006ae90 0x7f948006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.924+0000 7fe471ffb640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe46006af10 0x7fe46006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.924+0000 7fe471ffb640 1 -- 172.21.15.64:0/722903450 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4680896a0 con 0x7fe4740b30b0 2023-11-24T06:27:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.924+0000 7f9492ffd640 1 -- 172.21.15.64:0/3460749134 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9498088cb0 con 0x7f949c0ce6f0 2023-11-24T06:27:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.924+0000 7f94a1b03640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f948006ae90 0x7f948006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.925+0000 7fe4795a3640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe46006af10 0x7fe46006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.925+0000 7f94a1b03640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f948006ae90 0x7f948006d350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f949c0cd7a0 tx=0x7f948c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.926+0000 7fe4795a3640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe46006af10 0x7fe46006d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe47415cb40 tx=0x7fe46c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.928+0000 7f9492ffd640 1 -- 172.21.15.64:0/3460749134 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9498053d60 con 0x7f949c0ce6f0 2023-11-24T06:27:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f6feeffd640 1 -- 172.21.15.64:0/1444490816 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8031740 con 0x7f6ff00c6d80 2023-11-24T06:27:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f0b57fff640 1 -- 172.21.15.64:0/1805252355 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5803e610 con 0x7f0b60159f60 2023-11-24T06:27:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6015b050 con 0x7f0b60159f60 2023-11-24T06:27:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f6feeffd640 1 -- 172.21.15.64:0/1444490816 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd80318a0 con 0x7f6ff00c6d80 2023-11-24T06:27:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f6feeffd640 1 -- 172.21.15.64:0/1444490816 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8031c30 con 0x7f6ff00c6d80 2023-11-24T06:27:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7fa4e9d9b640 1 -- 172.21.15.64:0/2635971430 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e402cc60 con 0x7fa4ec15d320 2023-11-24T06:27:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7fa4e9d9b640 1 -- 172.21.15.64:0/2635971430 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e402cdc0 con 0x7fa4ec15d320 2023-11-24T06:27:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7fa4e9d9b640 1 -- 172.21.15.64:0/2635971430 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4034660 con 0x7fa4ec15d320 2023-11-24T06:27:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b600cd600 con 0x7f0b60159f60 2023-11-24T06:27:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f0b57fff640 1 -- 172.21.15.64:0/1805252355 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5803e790 con 0x7f0b60159f60 2023-11-24T06:27:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.934+0000 7f0b57fff640 1 -- 172.21.15.64:0/1805252355 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5803eb40 con 0x7f0b60159f60 2023-11-24T06:27:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.935+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/1444490816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff00c6d80 msgr2=0x7f6ff00d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.935+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/1444490816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00d1920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009870 tx=0x7f6fd802b860 comp rx=0 tx=0).stop 2023-11-24T06:27:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.935+0000 7fa4f113e640 1 -- 172.21.15.64:0/2635971430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec15d320 msgr2=0x7fa4ec06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b44005250 con 0x7f0b60159f60 2023-11-24T06:27:51.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7fe471ffb640 1 -- 172.21.15.64:0/722903450 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4680547e0 con 0x7fe4740b30b0 2023-11-24T06:27:51.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/1444490816 shutdown_connections 2023-11-24T06:27:51.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/1444490816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00d1920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/1444490816 >> 172.21.15.64:0/1444490816 conn(0x7f6ff008cc50 msgr2=0x7f6ff009c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.935+0000 7fa4f113e640 1 --2- 172.21.15.64:0/2635971430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec15d320 0x7fa4ec06cb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e402beb0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7fa4f113e640 1 -- 172.21.15.64:0/2635971430 shutdown_connections 2023-11-24T06:27:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7fa4f113e640 1 --2- 172.21.15.64:0/2635971430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec15d320 0x7fa4ec06cb80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7fa4f113e640 1 -- 172.21.15.64:0/2635971430 >> 172.21.15.64:0/2635971430 conn(0x7fa4ec0b0be0 msgr2=0x7fa4ec0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.937+0000 7fa4f113e640 1 -- 172.21.15.64:0/2635971430 shutdown_connections 2023-11-24T06:27:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/1444490816 shutdown_connections 2023-11-24T06:27:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.936+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/1444490816 wait complete. 2023-11-24T06:27:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.938+0000 7f0b57fff640 1 -- 172.21.15.64:0/1805252355 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b5803b050 con 0x7f0b60159f60 2023-11-24T06:27:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.937+0000 7fa4f113e640 1 -- 172.21.15.64:0/2635971430 wait complete. 2023-11-24T06:27:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.939+0000 7f6ff5ab4640 1 Processor -- start 2023-11-24T06:27:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.939+0000 7f6ff5ab4640 1 -- start start 2023-11-24T06:27:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.940+0000 7f0b57fff640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0b4006ae70 0x7f0b4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.940+0000 7f0b57fff640 1 -- 172.21.15.64:0/1805252355 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b580abd90 con 0x7f0b60159f60 2023-11-24T06:27:51.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.941+0000 7f0b5effd640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0b4006ae70 0x7f0b4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.941+0000 7fa4f113e640 1 Processor -- start 2023-11-24T06:27:51.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.941+0000 7f6ff5ab4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.942+0000 7fa4f113e640 1 -- start start 2023-11-24T06:27:51.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.942+0000 7f6ff5ab4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ff00d1540 0x7f6ff0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.942+0000 7f6ff5ab4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff0161610 0x7f6ff00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.942+0000 7f0b5effd640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0b4006ae70 0x7f0b4006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b60168540 tx=0x7f0b50007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.942+0000 7f6ff5ab4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0156d40 con 0x7f6ff00c6d80 2023-11-24T06:27:51.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.942+0000 7f6ff5ab4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0161af0 con 0x7f6ff0161610 2023-11-24T06:27:51.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.942+0000 7f6ff5ab4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0161dd0 con 0x7f6ff00d1540 2023-11-24T06:27:51.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.943+0000 7f6fef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.943+0000 7f6fef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48576/0 (socket says 172.21.15.64:48576) 2023-11-24T06:27:51.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.943+0000 7f6feffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff0161610 0x7f6ff00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.943+0000 7f6feffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff0161610 0x7f6ff00c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39744/0 (socket says 172.21.15.64:39744) 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.943+0000 7f6feffff640 1 -- 172.21.15.64:0/3396451469 learned_addr learned my addr 172.21.15.64:0/3396451469 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.943+0000 7f6fe7fff640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ff00d1540 0x7f6ff0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.944+0000 7f6fef7fe640 1 -- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ff00d1540 msgr2=0x7f6ff0156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.944+0000 7f6fef7fe640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ff00d1540 0x7f6ff0156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.944+0000 7f6fef7fe640 1 -- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff0161610 msgr2=0x7f6ff00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.944+0000 7f6fef7fe640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff0161610 0x7f6ff00c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.944+0000 7f6fef7fe640 1 -- 172.21.15.64:0/3396451469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd80094d0 con 0x7f6ff00c6d80 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.944+0000 7f6fe7fff640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ff00d1540 0x7f6ff0156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:51.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.944+0000 7f6feffff640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff0161610 0x7f6ff00c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:51.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.945+0000 7fa4f113e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec153950 0x7fa4ec0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.945+0000 7fa4f113e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4ec15ce90 0x7fa4ec1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.945+0000 7fa4f113e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4ec15d320 0x7fa4ec15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.945+0000 7fa4f113e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cddd0 con 0x7fa4ec15ce90 2023-11-24T06:27:51.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.945+0000 7fa4f113e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cc210 con 0x7fa4ec153950 2023-11-24T06:27:51.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.945+0000 7fa4f113e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec15aaf0 con 0x7fa4ec15d320 2023-11-24T06:27:51.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec153950 0x7fa4ec0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec153950 0x7fa4ec0cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39746/0 (socket says 172.21.15.64:39746) 2023-11-24T06:27:51.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7fa4ead9d640 1 -- 172.21.15.64:0/704315890 learned_addr learned my addr 172.21.15.64:0/704315890 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.945+0000 7f6fef7fe640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00cc3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c780 tx=0x7f6fe000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7f6fecff9640 1 -- 172.21.15.64:0/3396451469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0005270 con 0x7f6ff00c6d80 2023-11-24T06:27:51.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7f6fecff9640 1 -- 172.21.15.64:0/3396451469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe00053d0 con 0x7f6ff00c6d80 2023-11-24T06:27:51.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7f6fecff9640 1 -- 172.21.15.64:0/3396451469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000a880 con 0x7f6ff00c6d80 2023-11-24T06:27:51.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7fa4ea59c640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4ec15ce90 0x7fa4ec1585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7fa4eb59e640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4ec15d320 0x7fa4ec15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.946+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff01620b0 con 0x7f6ff00c6d80 2023-11-24T06:27:51.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.947+0000 7fa4ead9d640 1 -- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4ec15d320 msgr2=0x7fa4ec15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.947+0000 7fa4ead9d640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4ec15d320 0x7fa4ec15b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.947+0000 7fa4ead9d640 1 -- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4ec15ce90 msgr2=0x7fa4ec1585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.947+0000 7fa4ead9d640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4ec15ce90 0x7fa4ec1585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.947+0000 7fa4ead9d640 1 -- 172.21.15.64:0/704315890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4009a70 con 0x7fa4ec153950 2023-11-24T06:27:51.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.947+0000 7fa4ea59c640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4ec15ce90 0x7fa4ec1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:51.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.948+0000 7fa4ead9d640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec153950 0x7fa4ec0cbb10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc00c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.947+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff00b6250 con 0x7f6ff00c6d80 2023-11-24T06:27:51.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.949+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff00b64a0 con 0x7f6ff00c6d80 2023-11-24T06:27:51.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.950+0000 7f6fecff9640 1 -- 172.21.15.64:0/3396451469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fe000a9e0 con 0x7f6ff00c6d80 2023-11-24T06:27:51.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.952+0000 7f6fecff9640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6fd006ae90 0x7f6fd006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.952+0000 7f6fecff9640 1 -- 172.21.15.64:0/3396451469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fe0088b80 con 0x7f6ff00c6d80 2023-11-24T06:27:51.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.953+0000 7f6fe7fff640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6fd006ae90 0x7f6fd006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.954+0000 7fa4d37fe640 1 -- 172.21.15.64:0/704315890 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc005d00 con 0x7fa4ec153950 2023-11-24T06:27:51.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.954+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ec15add0 con 0x7fa4ec153950 2023-11-24T06:27:51.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.954+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ec0cc4f0 con 0x7fa4ec153950 2023-11-24T06:27:51.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.956+0000 7f0b57fff640 1 -- 172.21.15.64:0/1805252355 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b58076da0 con 0x7f0b60159f60 2023-11-24T06:27:51.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.955+0000 7fa4d37fe640 1 -- 172.21.15.64:0/704315890 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc005e80 con 0x7fa4ec153950 2023-11-24T06:27:51.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.956+0000 7fa4d37fe640 1 -- 172.21.15.64:0/704315890 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc007310 con 0x7fa4ec153950 2023-11-24T06:27:51.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.956+0000 7f6fe7fff640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6fd006ae90 0x7f6fd006d350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8035000 tx=0x7f6fd8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.956+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4cc005250 con 0x7fa4ec153950 2023-11-24T06:27:51.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.959+0000 7fa4d37fe640 1 -- 172.21.15.64:0/704315890 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4dc007470 con 0x7fa4ec153950 2023-11-24T06:27:51.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.961+0000 7fa4d37fe640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa4d806af10 0x7fa4d806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.961+0000 7f6fecff9640 1 -- 172.21.15.64:0/3396451469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fe0053e80 con 0x7f6ff00c6d80 2023-11-24T06:27:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.961+0000 7fa4d37fe640 1 -- 172.21.15.64:0/704315890 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4dc088e00 con 0x7fa4ec153950 2023-11-24T06:27:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.962+0000 7fa4ea59c640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa4d806af10 0x7fa4d806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.963+0000 7fa4ea59c640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa4d806af10 0x7fa4d806d3d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec1f6700 tx=0x7fa4e0006050 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.974+0000 7fa4d37fe640 1 -- 172.21.15.64:0/704315890 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa4dc053f40 con 0x7fa4ec153950 2023-11-24T06:27:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.984+0000 7f3ac446b640 1 Processor -- start 2023-11-24T06:27:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.984+0000 7f3ac446b640 1 -- start start 2023-11-24T06:27:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.986+0000 7f3ac446b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3abc1536e0 0x7f3abc15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.986+0000 7f3ac446b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3abc0b2e50 con 0x7f3abc0b16d0 2023-11-24T06:27:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.987+0000 7f3ac446b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3abc0b3130 con 0x7f3abc153b70 2023-11-24T06:27:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.987+0000 7f3ac1a06640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3abc1536e0 0x7f3abc15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.987+0000 7f3ac446b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3abc160210 con 0x7f3abc1536e0 2023-11-24T06:27:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.987+0000 7f3ac1a06640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3abc1536e0 0x7f3abc15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48598/0 (socket says 172.21.15.64:48598) 2023-11-24T06:27:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.987+0000 7f3ac1a06640 1 -- 172.21.15.64:0/2408762175 learned_addr learned my addr 172.21.15.64:0/2408762175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.986+0000 7f087f47c640 1 Processor -- start 2023-11-24T06:27:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.987+0000 7f087f47c640 1 -- start start 2023-11-24T06:27:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.988+0000 7f3ac1a06640 1 -- 172.21.15.64:0/2408762175 >> v1:172.21.15.162:6789/0 conn(0x7f3abc153b70 legacy=0x7f3abc06c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.988+0000 7f3ac1a06640 1 -- 172.21.15.64:0/2408762175 >> v1:172.21.15.64:6789/0 conn(0x7f3abc0b16d0 legacy=0x7f3abc0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f3ac1a06640 1 -- 172.21.15.64:0/2408762175 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc0b3410 con 0x7f3abc1536e0 2023-11-24T06:27:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f087f47c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08780d0280 0x7f08780ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f087f47c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f08780ce6f0 0x7f08780b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f087f47c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08780b6750 con 0x7f087815d590 2023-11-24T06:27:51.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f087f47c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08780cf1f0 con 0x7f08780ce6f0 2023-11-24T06:27:51.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f087f47c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08780cef10 con 0x7f08780d0280 2023-11-24T06:27:51.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f087d218640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f087815d590 0x7f08780cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44862/0 (socket says 172.21.15.64:44862) 2023-11-24T06:27:51.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.989+0000 7f087d218640 1 -- 172.21.15.64:0/924362998 learned_addr learned my addr 172.21.15.64:0/924362998 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:51.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.990+0000 7f087da19640 1 --2- 172.21.15.64:0/924362998 >> v2:172.21.15.64:3300/0 conn(0x7f08780ce6f0 0x7f08780b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.991+0000 7f087da19640 1 -- 172.21.15.64:0/924362998 >> v2:172.21.15.162:3300/0 conn(0x7f08780d0280 msgr2=0x7f08780ce1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:27:51.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.991+0000 7f087da19640 1 --2- 172.21.15.64:0/924362998 >> v2:172.21.15.162:3300/0 conn(0x7f08780d0280 0x7f08780ce1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.991+0000 7f087da19640 1 -- 172.21.15.64:0/924362998 >> v1:172.21.15.64:6789/0 conn(0x7f087815d590 legacy=0x7f08780cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.991+0000 7f087da19640 1 -- 172.21.15.64:0/924362998 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780b6100 con 0x7f08780ce6f0 2023-11-24T06:27:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.991+0000 7f3ac1a06640 1 --2- 172.21.15.64:0/2408762175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc1536e0 0x7f3abc15ce90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3abc15df50 tx=0x7f3aac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256bdbd182aa68be server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.993+0000 7f087da19640 1 --2- 172.21.15.64:0/924362998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08780ce6f0 0x7f08780b6be0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f087808d970 tx=0x7f086802b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dc78a67b78679a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.994+0000 7f086ffff640 1 -- 172.21.15.64:0/924362998 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868030710 con 0x7f08780ce6f0 2023-11-24T06:27:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.994+0000 7f086ffff640 1 -- 172.21.15.64:0/924362998 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086802e050 con 0x7f08780ce6f0 2023-11-24T06:27:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.994+0000 7f086ffff640 1 -- 172.21.15.64:0/924362998 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868030a90 con 0x7f08780ce6f0 2023-11-24T06:27:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f3ac1205640 1 -- 172.21.15.64:0/2408762175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac030840 con 0x7f3abc1536e0 2023-11-24T06:27:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f087f47c640 1 -- 172.21.15.64:0/924362998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08780ce6f0 msgr2=0x7f08780b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f087f47c640 1 --2- 172.21.15.64:0/924362998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08780ce6f0 0x7f08780b6be0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f087808d970 tx=0x7f086802b730 comp rx=0 tx=0).stop 2023-11-24T06:27:51.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f3ac1205640 1 -- 172.21.15.64:0/2408762175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac0309a0 con 0x7f3abc1536e0 2023-11-24T06:27:51.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f3ac1205640 1 -- 172.21.15.64:0/2408762175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac030d70 con 0x7f3abc1536e0 2023-11-24T06:27:51.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f087f47c640 1 -- 172.21.15.64:0/924362998 shutdown_connections 2023-11-24T06:27:51.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f087f47c640 1 --2- 172.21.15.64:0/924362998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08780ce6f0 0x7f08780b6be0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f087f47c640 1 --2- 172.21.15.64:0/924362998 >> v2:172.21.15.162:3300/0 conn(0x7f08780d0280 0x7f08780ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.995+0000 7f087f47c640 1 -- 172.21.15.64:0/924362998 >> 172.21.15.64:0/924362998 conn(0x7f087809c540 msgr2=0x7f08780d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f3ac446b640 1 -- 172.21.15.64:0/2408762175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc1536e0 msgr2=0x7f3abc15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f3ac446b640 1 --2- 172.21.15.64:0/2408762175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc1536e0 0x7f3abc15ce90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3abc15df50 tx=0x7f3aac02b980 comp rx=0 tx=0).stop 2023-11-24T06:27:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f087f47c640 1 -- 172.21.15.64:0/924362998 shutdown_connections 2023-11-24T06:27:52.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f3ac446b640 1 -- 172.21.15.64:0/2408762175 shutdown_connections 2023-11-24T06:27:52.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f3ac446b640 1 --2- 172.21.15.64:0/2408762175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc1536e0 0x7f3abc15ce90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f3ac446b640 1 -- 172.21.15.64:0/2408762175 >> 172.21.15.64:0/2408762175 conn(0x7f3abc0b0be0 msgr2=0x7f3abc0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f3ac446b640 1 -- 172.21.15.64:0/2408762175 shutdown_connections 2023-11-24T06:27:52.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f3ac446b640 1 -- 172.21.15.64:0/2408762175 wait complete. 2023-11-24T06:27:52.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.996+0000 7f087f47c640 1 -- 172.21.15.64:0/924362998 wait complete. 2023-11-24T06:27:52.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.999+0000 7f087f47c640 1 Processor -- start 2023-11-24T06:27:52.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.999+0000 7f3ac446b640 1 Processor -- start 2023-11-24T06:27:52.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.999+0000 7f087f47c640 1 -- start start 2023-11-24T06:27:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:51.999+0000 7f3ac446b640 1 -- start start 2023-11-24T06:27:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.001+0000 7f3ac446b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc0b16d0 0x7f3abc149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.001+0000 7f087f47c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780ce6f0 0x7f087815c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.001+0000 7f3ac2207640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc0b16d0 0x7f3abc149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.001+0000 7f3ac2207640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc0b16d0 0x7f3abc149fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48614/0 (socket says 172.21.15.64:48614) 2023-11-24T06:27:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f3ac446b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3abc14a4e0 0x7f3abc14a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f087f47c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08780d0280 0x7f08781f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f3ac2207640 1 -- 172.21.15.64:0/1215839835 learned_addr learned my addr 172.21.15.64:0/1215839835 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:52.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f3ac446b640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3abc15b270 0x7f3abc14ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc1473d0 con 0x7f3abc0b16d0 2023-11-24T06:27:52.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f087f47c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f087815d590 0x7f08780c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc15c3f0 con 0x7f3abc15b270 2023-11-24T06:27:52.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc15b7e0 con 0x7f3abc14a4e0 2023-11-24T06:27:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f3ac2a08640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3abc15b270 0x7f3abc14ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f087da19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f087815d590 0x7f08780c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f087da19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f087815d590 0x7f08780c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48616/0 (socket says 172.21.15.64:48616) 2023-11-24T06:27:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f087d218640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780ce6f0 0x7f087815c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.002+0000 7f087d218640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780ce6f0 0x7f087815c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:39754/0 (socket says 172.21.15.64:39754) 2023-11-24T06:27:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f087da19640 1 -- 172.21.15.64:0/2743889831 learned_addr learned my addr 172.21.15.64:0/2743889831 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac2207640 1 -- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3abc14a4e0 msgr2=0x7f3abc14a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac1a06640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3abc14a4e0 0x7f3abc14a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781f3260 con 0x7f087815d590 2023-11-24T06:27:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f087ca17640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08780d0280 0x7f08781f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac2207640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3abc14a4e0 0x7f3abc14a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac2207640 1 -- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3abc15b270 msgr2=0x7f3abc14ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac2207640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3abc15b270 0x7f3abc14ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac2207640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0094d0 con 0x7f3abc0b16d0 2023-11-24T06:27:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780cbd40 con 0x7f08780ce6f0 2023-11-24T06:27:52.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac2a08640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3abc15b270 0x7f3abc14ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:52.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08780cbfe0 con 0x7f08780d0280 2023-11-24T06:27:52.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac1a06640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3abc14a4e0 0x7f3abc14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:52.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.003+0000 7f3ac2207640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc0b16d0 0x7f3abc149fa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c430 tx=0x7f3ab800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f3ab2ffd640 1 -- 172.21.15.64:0/1215839835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8011790 con 0x7f3abc0b16d0 2023-11-24T06:27:52.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f3ab2ffd640 1 -- 172.21.15.64:0/1215839835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab80118f0 con 0x7f3abc0b16d0 2023-11-24T06:27:52.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f3ab2ffd640 1 -- 172.21.15.64:0/1215839835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8011ce0 con 0x7f3abc0b16d0 2023-11-24T06:27:52.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc15bac0 con 0x7f3abc0b16d0 2023-11-24T06:27:52.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f087da19640 1 -- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08780d0280 msgr2=0x7f08781f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f087da19640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08780d0280 0x7f08781f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f087da19640 1 -- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780ce6f0 msgr2=0x7f087815c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f087da19640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780ce6f0 0x7f087815c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f087da19640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009400 con 0x7f087815d590 2023-11-24T06:27:52.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f087d218640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780ce6f0 0x7f087815c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:52.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.005+0000 7f087da19640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f087815d590 0x7f08780c6ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f086802bd40 tx=0x7f08680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.006+0000 7f086dffb640 1 -- 172.21.15.64:0/2743889831 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680329d0 con 0x7f087815d590 2023-11-24T06:27:52.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.006+0000 7f086dffb640 1 -- 172.21.15.64:0/2743889831 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0868032b30 con 0x7f087815d590 2023-11-24T06:27:52.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.006+0000 7f086dffb640 1 -- 172.21.15.64:0/2743889831 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086803d480 con 0x7f087815d590 2023-11-24T06:27:52.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.004+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc15c080 con 0x7f3abc0b16d0 2023-11-24T06:27:52.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.007+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3abc15c210 con 0x7f3abc0b16d0 2023-11-24T06:27:52.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.006+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08780cc2c0 con 0x7f087815d590 2023-11-24T06:27:52.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.006+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08780cc880 con 0x7f087815d590 2023-11-24T06:27:52.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.009+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08780ccaa0 con 0x7f087815d590 2023-11-24T06:27:52.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.015+0000 7f3ab2ffd640 1 -- 172.21.15.64:0/1215839835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ab801c460 con 0x7f3abc0b16d0 2023-11-24T06:27:52.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.011+0000 7f086dffb640 1 -- 172.21.15.64:0/2743889831 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0868039040 con 0x7f087815d590 2023-11-24T06:27:52.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.013+0000 7f086dffb640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f085c06af10 0x7f085c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.013+0000 7f086dffb640 1 -- 172.21.15.64:0/2743889831 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08680ab640 con 0x7f087815d590 2023-11-24T06:27:52.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.014+0000 7f087d218640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f085c06af10 0x7f085c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.015+0000 7f087d218640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f085c06af10 0x7f085c06d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0870001330 tx=0x7f08700017a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.016+0000 7f3ab2ffd640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3aa006ae90 0x7f3aa006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.017+0000 7f3ab2ffd640 1 -- 172.21.15.64:0/1215839835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ab8089580 con 0x7f3abc0b16d0 2023-11-24T06:27:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.018+0000 7f3ac1a06640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3aa006ae90 0x7f3aa006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.020+0000 7f3ac1a06640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3aa006ae90 0x7f3aa006d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3aac036000 tx=0x7f3aac03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.020+0000 7f3ab2ffd640 1 -- 172.21.15.64:0/1215839835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ab8054620 con 0x7f3abc0b16d0 2023-11-24T06:27:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.028+0000 7f086dffb640 1 -- 172.21.15.64:0/2743889831 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08680768a0 con 0x7f087815d590 2023-11-24T06:27:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.310+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c580ce530 con 0x7f6c580b22b0 2023-11-24T06:27:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.311+0000 7f6c557fa640 1 -- 172.21.15.64:0/1303706169 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c4c035070 con 0x7f6c580b22b0 2023-11-24T06:27:52.315 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-11-24T06:27:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.315+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6c4406aea0 msgr2=0x7f6c4406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.315+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6c4406aea0 0x7f6c4406d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c48006b10 tx=0x7f6c48008040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.315+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c580b22b0 msgr2=0x7f6c580c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.315+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c580b22b0 0x7f6c580c5c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0090b0 tx=0x7f6c4c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.316+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 shutdown_connections 2023-11-24T06:27:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.316+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6c4406aea0 0x7f6c4406d360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.316+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c5815d560 0x7f6c580b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.316+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c580cff70 0x7f6c580b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.316+0000 7f6c5f0e7640 1 --2- 172.21.15.64:0/1303706169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c580b22b0 0x7f6c580c5c70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.316+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 >> 172.21.15.64:0/1303706169 conn(0x7f6c580d3020 msgr2=0x7f6c58152330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.317+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 shutdown_connections 2023-11-24T06:27:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.317+0000 7f6c5f0e7640 1 -- 172.21.15.64:0/1303706169 wait complete. 2023-11-24T06:27:52.342 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:52.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.335+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9484005500 con 0x7f949c0ce6f0 2023-11-24T06:27:52.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.336+0000 7f9492ffd640 1 -- 172.21.15.64:0/3460749134 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9498017020 con 0x7f949c0ce6f0 2023-11-24T06:27:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.340+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f948006ae90 msgr2=0x7f948006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.340+0000 7f94a4568640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f948006ae90 0x7f948006d350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f949c0cd7a0 tx=0x7f948c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.340+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 msgr2=0x7f949c1f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.340+0000 7f94a4568640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c1f5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f949800c430 tx=0x7f949800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 shutdown_connections 2023-11-24T06:27:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f948006ae90 0x7f948006d350 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f949c0cd7a0 tx=0x7f948c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f949c15d590 0x7f949c0c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f949c0d0280 0x7f949c0c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 --2- 172.21.15.64:0/3460749134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f949c0ce6f0 0x7f949c1f5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 >> 172.21.15.64:0/3460749134 conn(0x7f949c09c540 msgr2=0x7f949c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 shutdown_connections 2023-11-24T06:27:52.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.341+0000 7f94a4568640 1 -- 172.21.15.64:0/3460749134 wait complete. 2023-11-24T06:27:52.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2023-11-24T06:27:52.349 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.358+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe4740675d0 con 0x7fe4740b30b0 2023-11-24T06:27:52.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.359+0000 7fe471ffb640 1 -- 172.21.15.64:0/722903450 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe468018090 con 0x7fe4740b30b0 2023-11-24T06:27:52.364 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:52.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe46006af10 msgr2=0x7fe46006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe46006af10 0x7fe46006d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe47415cb40 tx=0x7fe46c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4740b30b0 msgr2=0x7fe47406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4740b30b0 0x7fe47406d180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c7c0 tx=0x7fe46800cc80 comp rx=0 tx=0).stop 2023-11-24T06:27:52.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 shutdown_connections 2023-11-24T06:27:52.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe46006af10 0x7fe46006d3d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe47415d460 0x7fe47406d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4740b30b0 0x7fe47406d180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 --2- 172.21.15.64:0/722903450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47406e380 0x7fe474152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.363+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 >> 172.21.15.64:0/722903450 conn(0x7fe4740b0be0 msgr2=0x7fe4740ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.364+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 shutdown_connections 2023-11-24T06:27:52.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.364+0000 7fe47b807640 1 -- 172.21.15.64:0/722903450 wait complete. 2023-11-24T06:27:52.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2023-11-24T06:27:52.372 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.373+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0b44005500 con 0x7f0b60159f60 2023-11-24T06:27:52.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.374+0000 7f0b57fff640 1 -- 172.21.15.64:0/1805252355 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b58039090 con 0x7f0b60159f60 2023-11-24T06:27:52.377 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.378+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0b4006ae70 msgr2=0x7f0b4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.378+0000 7f0b66190640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0b4006ae70 0x7f0b4006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b60168540 tx=0x7f0b50007a40 comp rx=0 tx=0).stop 2023-11-24T06:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.378+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b60159f60 msgr2=0x7f0b60150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.378+0000 7f0b66190640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b60159f60 0x7f0b60150fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b58037040 tx=0x7f0b58004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa4cc0057d0 con 0x7fa4ec153950 2023-11-24T06:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 shutdown_connections 2023-11-24T06:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0b4006ae70 0x7f0b4006d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b601514e0 0x7f0b60159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6015dd30 0x7f0b60158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 --2- 172.21.15.64:0/1805252355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b60159f60 0x7f0b60150fa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 >> 172.21.15.64:0/1805252355 conn(0x7f0b600cf700 msgr2=0x7f0b60153320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 shutdown_connections 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.379+0000 7f0b66190640 1 -- 172.21.15.64:0/1805252355 wait complete. 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.380+0000 7fa4d37fe640 1 -- 172.21.15.64:0/704315890 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa4dc017090 con 0x7fa4ec153950 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:52.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.382+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6ff00675d0 con 0x7f6ff00c6d80 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa4d806af10 msgr2=0x7fa4d806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa4d806af10 0x7fa4d806d3d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec1f6700 tx=0x7fa4e0006050 comp rx=0 tx=0).stop 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec153950 msgr2=0x7fa4ec0cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec153950 0x7fa4ec0cbb10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc00c870 comp rx=0 tx=0).stop 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 shutdown_connections 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa4d806af10 0x7fa4d806d3d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4ec15d320 0x7fa4ec15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4ec15ce90 0x7fa4ec1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 --2- 172.21.15.64:0/704315890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4ec153950 0x7fa4ec0cbb10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 >> 172.21.15.64:0/704315890 conn(0x7fa4ec0b0be0 msgr2=0x7fa4ec14ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7f6fecff9640 1 -- 172.21.15.64:0/3396451469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6fe0017090 con 0x7f6ff00c6d80 2023-11-24T06:27:52.388 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-11-24T06:27:52.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 shutdown_connections 2023-11-24T06:27:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.384+0000 7fa4f113e640 1 -- 172.21.15.64:0/704315890 wait complete. 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.386+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6fd006ae90 msgr2=0x7f6fd006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.386+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6fd006ae90 0x7f6fd006d350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8035000 tx=0x7f6fd8008030 comp rx=0 tx=0).stop 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.386+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff00c6d80 msgr2=0x7f6ff00cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.386+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00cc3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c780 tx=0x7f6fe000cc40 comp rx=0 tx=0).stop 2023-11-24T06:27:52.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.387+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 shutdown_connections 2023-11-24T06:27:52.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.387+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6fd006ae90 0x7f6fd006d350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.387+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ff0161610 0x7f6ff00c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.387+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ff00d1540 0x7f6ff0156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.387+0000 7f6ff5ab4640 1 --2- 172.21.15.64:0/3396451469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff00c6d80 0x7f6ff00cc3f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2023-11-24T06:27:52.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.387+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 >> 172.21.15.64:0/3396451469 conn(0x7f6ff008cc50 msgr2=0x7f6ff006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.387+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 shutdown_connections 2023-11-24T06:27:52.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.388+0000 7f6ff5ab4640 1 -- 172.21.15.64:0/3396451469 wait complete. 2023-11-24T06:27:52.395 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2023-11-24T06:27:52.412 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2023-11-24T06:27:52.413 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2023-11-24T06:27:52.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.411+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f08780675d0 con 0x7f087815d590 2023-11-24T06:27:52.416 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.411+0000 7f086dffb640 1 -- 172.21.15.64:0/2743889831 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0868079ec0 con 0x7f087815d590 2023-11-24T06:27:52.417 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-11-24T06:27:52.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f085c06af10 msgr2=0x7f085c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f085c06af10 0x7f085c06d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0870001330 tx=0x7f08700017a0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f087815d590 msgr2=0x7f08780c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f087815d590 0x7f08780c6ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f086802bd40 tx=0x7f08680046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 shutdown_connections 2023-11-24T06:27:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f085c06af10 0x7f085c06d3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f087815d590 0x7f08780c6ad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08780d0280 0x7f08781f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 --2- 172.21.15.64:0/2743889831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780ce6f0 0x7f087815c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 >> 172.21.15.64:0/2743889831 conn(0x7f087809c540 msgr2=0x7f087815fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 shutdown_connections 2023-11-24T06:27:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.414+0000 7f087f47c640 1 -- 172.21.15.64:0/2743889831 wait complete. 2023-11-24T06:27:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.421+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3abc0675d0 con 0x7f3abc0b16d0 2023-11-24T06:27:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.422+0000 7f3ab2ffd640 1 -- 172.21.15.64:0/1215839835 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ab805e020 con 0x7f3abc0b16d0 2023-11-24T06:27:52.426 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3aa006ae90 msgr2=0x7f3aa006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3aa006ae90 0x7f3aa006d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3aac036000 tx=0x7f3aac03b040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc0b16d0 msgr2=0x7f3abc149fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc0b16d0 0x7f3abc149fa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c430 tx=0x7f3ab800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 shutdown_connections 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3aa006ae90 0x7f3aa006d350 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3abc15b270 0x7f3abc14ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3abc14a4e0 0x7f3abc14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 --2- 172.21.15.64:0/1215839835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3abc0b16d0 0x7f3abc149fa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 >> 172.21.15.64:0/1215839835 conn(0x7f3abc0b0be0 msgr2=0x7f3abc15fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 shutdown_connections 2023-11-24T06:27:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.424+0000 7f3ac446b640 1 -- 172.21.15.64:0/1215839835 wait complete. 2023-11-24T06:27:52.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2023-11-24T06:27:52.434 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2023-11-24T06:27:52.445 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:52.445 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:27:52.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:52.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.707+0000 7ff6a6dc9640 1 Processor -- start 2023-11-24T06:27:52.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.707+0000 7ff6a6dc9640 1 -- start start 2023-11-24T06:27:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.709+0000 7ff6a6dc9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6a013df80 0x7ff6a015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.709+0000 7ff6a6dc9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6a00cf9a0 0x7ff6a00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.709+0000 7ff6a6dc9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a006c470 con 0x7ff6a013e360 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.709+0000 7ff6a6dc9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a006c750 con 0x7ff6a00cf9a0 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.709+0000 7ff6a6dc9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a006ca30 con 0x7ff6a013df80 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a5366640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6a00cf9a0 0x7ff6a00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a5366640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6a00cf9a0 0x7ff6a00cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48632/0 (socket says 172.21.15.64:48632) 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a5366640 1 -- 172.21.15.64:0/3669725890 learned_addr learned my addr 172.21.15.64:0/3669725890 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a4b65640 1 --2- 172.21.15.64:0/3669725890 >> v2:172.21.15.162:3300/0 conn(0x7ff6a013df80 0x7ff6a015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a5366640 1 -- 172.21.15.64:0/3669725890 >> v2:172.21.15.162:3300/0 conn(0x7ff6a013df80 msgr2=0x7ff6a015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a5366640 1 --2- 172.21.15.64:0/3669725890 >> v2:172.21.15.162:3300/0 conn(0x7ff6a013df80 0x7ff6a015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a5366640 1 -- 172.21.15.64:0/3669725890 >> v1:172.21.15.162:6789/0 conn(0x7ff6a013e360 legacy=0x7ff6a00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.710+0000 7ff6a5366640 1 -- 172.21.15.64:0/3669725890 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a006cd10 con 0x7ff6a00cf9a0 2023-11-24T06:27:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.711+0000 7ff6a5366640 1 --2- 172.21.15.64:0/3669725890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a00cf9a0 0x7ff6a00cd840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0072770 tx=0x7ff69802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceca6fbaddb461a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff69f7fe640 1 -- 172.21.15.64:0/3669725890 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69802ca80 con 0x7ff6a00cf9a0 2023-11-24T06:27:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff69f7fe640 1 -- 172.21.15.64:0/3669725890 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69802cbe0 con 0x7ff6a00cf9a0 2023-11-24T06:27:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff69f7fe640 1 -- 172.21.15.64:0/3669725890 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980343f0 con 0x7ff6a00cf9a0 2023-11-24T06:27:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/3669725890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a00cf9a0 msgr2=0x7ff6a00cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/3669725890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a00cf9a0 0x7ff6a00cd840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0072770 tx=0x7ff69802bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/3669725890 shutdown_connections 2023-11-24T06:27:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/3669725890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a00cf9a0 0x7ff6a00cd840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/3669725890 >> v2:172.21.15.162:3300/0 conn(0x7ff6a013df80 0x7ff6a015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/3669725890 >> 172.21.15.64:0/3669725890 conn(0x7ff6a006a800 msgr2=0x7ff6a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/3669725890 shutdown_connections 2023-11-24T06:27:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.712+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/3669725890 wait complete. 2023-11-24T06:27:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.714+0000 7ff6a6dc9640 1 Processor -- start 2023-11-24T06:27:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.715+0000 7ff6a6dc9640 1 -- start start 2023-11-24T06:27:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a6dc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00cf9a0 0x7ff6a0137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a6dc9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a013df80 0x7ff6a013c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a6dc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a013e360 0x7ff6a013cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a6dc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0139660 con 0x7ff6a013e360 2023-11-24T06:27:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a6dc9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a01f1400 con 0x7ff6a013df80 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a6dc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a01f16e0 con 0x7ff6a00cf9a0 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a013e360 0x7ff6a013cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a013e360 0x7ff6a013cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48634/0 (socket says 172.21.15.64:48634) 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 -- 172.21.15.64:0/363454141 learned_addr learned my addr 172.21.15.64:0/363454141 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff69ffff640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a013df80 0x7ff6a013c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a4b65640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00cf9a0 0x7ff6a0137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 -- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00cf9a0 msgr2=0x7ff6a0137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00cf9a0 0x7ff6a0137900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 -- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a013df80 msgr2=0x7ff6a013c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a013df80 0x7ff6a013c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.716+0000 7ff6a5366640 1 -- 172.21.15.64:0/363454141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009b30 con 0x7ff6a013e360 2023-11-24T06:27:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.717+0000 7ff6a5366640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a013e360 0x7ff6a013cbc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff698036040 tx=0x7ff698004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.717+0000 7ff69d7fa640 1 -- 172.21.15.64:0/363454141 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69803e450 con 0x7ff6a013e360 2023-11-24T06:27:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.717+0000 7ff69d7fa640 1 -- 172.21.15.64:0/363454141 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69803e5d0 con 0x7ff6a013e360 2023-11-24T06:27:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.717+0000 7ff69d7fa640 1 -- 172.21.15.64:0/363454141 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69803e980 con 0x7ff6a013e360 2023-11-24T06:27:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.717+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a01f19c0 con 0x7ff6a013e360 2023-11-24T06:27:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.717+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a0137e40 con 0x7ff6a013e360 2023-11-24T06:27:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.718+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a00675d0 con 0x7ff6a013e360 2023-11-24T06:27:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.719+0000 7ff69d7fa640 1 -- 172.21.15.64:0/363454141 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff69802cd50 con 0x7ff6a013e360 2023-11-24T06:27:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.720+0000 7ff69d7fa640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff68406ae40 0x7ff68406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.720+0000 7ff69d7fa640 1 -- 172.21.15.64:0/363454141 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6980ac1b0 con 0x7ff6a013e360 2023-11-24T06:27:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.720+0000 7ff6a4b65640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff68406ae40 0x7ff68406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.721+0000 7ff6a4b65640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff68406ae40 0x7ff68406d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff694006fd0 tx=0x7ff694008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.726+0000 7ff69d7fa640 1 -- 172.21.15.64:0/363454141 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6980774c0 con 0x7ff6a013e360 2023-11-24T06:27:52.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.910+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6a015f9c0 con 0x7ff68406ae40 2023-11-24T06:27:52.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.918+0000 7ff69d7fa640 1 -- 172.21.15.64:0/363454141 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264350 (secure 0 0 0) 0x7ff69400c040 con 0x7ff68406ae40 2023-11-24T06:27:52.927 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:52.929 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:27:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff68406ae40 msgr2=0x7ff68406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff68406ae40 0x7ff68406d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff694006fd0 tx=0x7ff694008040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a013e360 msgr2=0x7ff6a013cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a013e360 0x7ff6a013cbc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff698036040 tx=0x7ff698004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:52.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 shutdown_connections 2023-11-24T06:27:52.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff68406ae40 0x7ff68406d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a013e360 0x7ff6a013cbc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a013df80 0x7ff6a013c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 --2- 172.21.15.64:0/363454141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00cf9a0 0x7ff6a0137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 >> 172.21.15.64:0/363454141 conn(0x7ff6a006a800 msgr2=0x7ff6a00b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 shutdown_connections 2023-11-24T06:27:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:52.923+0000 7ff6a6dc9640 1 -- 172.21.15.64:0/363454141 wait complete. 2023-11-24T06:27:52.946 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:27:51.104009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219328,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755392,"statfs":{"total":773094113280,"available":772869521408,"internally_reserved":0,"allocated":3055616,"data_stored":1815972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.754825"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658921+0000","last_change":"2023-11-24T06:27:42.394622+0000","last_active":"2023-11-24T06:27:43.658921+0000","last_peered":"2023-11-24T06:27:43.658921+0000","last_clean":"2023-11-24T06:27:43.658921+0000","last_became_active":"2023-11-24T06:27:42.394278+0000","last_became_peered":"2023-11-24T06:27:42.394278+0000","last_unstale":"2023-11-24T06:27:43.658921+0000","last_undegraded":"2023-11-24T06:27:43.658921+0000","last_fullsized":"2023-11-24T06:27:43.658921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659042+0000","last_change":"2023-11-24T06:27:42.395699+0000","last_active":"2023-11-24T06:27:43.659042+0000","last_peered":"2023-11-24T06:27:43.659042+0000","last_clean":"2023-11-24T06:27:43.659042+0000","last_became_active":"2023-11-24T06:27:42.395424+0000","last_became_peered":"2023-11-24T06:27:42.395424+0000","last_unstale":"2023-11-24T06:27:43.659042+0000","last_undegraded":"2023-11-24T06:27:43.659042+0000","last_fullsized":"2023-11-24T06:27:43.659042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660634+0000","last_change":"2023-11-24T06:27:42.390431+0000","last_active":"2023-11-24T06:27:43.660634+0000","last_peered":"2023-11-24T06:27:43.660634+0000","last_clean":"2023-11-24T06:27:43.660634+0000","last_became_active":"2023-11-24T06:27:42.390163+0000","last_became_peered":"2023-11-24T06:27:42.390163+0000","last_unstale":"2023-11-24T06:27:43.660634+0000","last_undegraded":"2023-11-24T06:27:43.660634+0000","last_fullsized":"2023-11-24T06:27:43.660634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357785+0000","last_change":"2023-11-24T06:27:42.393940+0000","last_active":"2023-11-24T06:27:43.357785+0000","last_peered":"2023-11-24T06:27:43.357785+0000","last_clean":"2023-11-24T06:27:43.357785+0000","last_became_active":"2023-11-24T06:27:42.393341+0000","last_became_peered":"2023-11-24T06:27:42.393341+0000","last_unstale":"2023-11-24T06:27:43.357785+0000","last_undegraded":"2023-11-24T06:27:43.357785+0000","last_fullsized":"2023-11-24T06:27:43.357785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658343+0000","last_change":"2023-11-24T06:27:42.392760+0000","last_active":"2023-11-24T06:27:43.658343+0000","last_peered":"2023-11-24T06:27:43.658343+0000","last_clean":"2023-11-24T06:27:43.658343+0000","last_became_active":"2023-11-24T06:27:42.390011+0000","last_became_peered":"2023-11-24T06:27:42.390011+0000","last_unstale":"2023-11-24T06:27:43.658343+0000","last_undegraded":"2023-11-24T06:27:43.658343+0000","last_fullsized":"2023-11-24T06:27:43.658343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355889+0000","last_change":"2023-11-24T06:27:42.397895+0000","last_active":"2023-11-24T06:27:43.355889+0000","last_peered":"2023-11-24T06:27:43.355889+0000","last_clean":"2023-11-24T06:27:43.355889+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355889+0000","last_undegraded":"2023-11-24T06:27:43.355889+0000","last_fullsized":"2023-11-24T06:27:43.355889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657379+0000","last_change":"2023-11-24T06:27:42.398329+0000","last_active":"2023-11-24T06:27:43.657379+0000","last_peered":"2023-11-24T06:27:43.657379+0000","last_clean":"2023-11-24T06:27:43.657379+0000","last_became_active":"2023-11-24T06:27:42.398095+0000","last_became_peered":"2023-11-24T06:27:42.398095+0000","last_unstale":"2023-11-24T06:27:43.657379+0000","last_undegraded":"2023-11-24T06:27:43.657379+0000","last_fullsized":"2023-11-24T06:27:43.657379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355033+0000","last_change":"2023-11-24T06:27:42.395940+0000","last_active":"2023-11-24T06:27:43.355033+0000","last_peered":"2023-11-24T06:27:43.355033+0000","last_clean":"2023-11-24T06:27:43.355033+0000","last_became_active":"2023-11-24T06:27:42.395595+0000","last_became_peered":"2023-11-24T06:27:42.395595+0000","last_unstale":"2023-11-24T06:27:43.355033+0000","last_undegraded":"2023-11-24T06:27:43.355033+0000","last_fullsized":"2023-11-24T06:27:43.355033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659266+0000","last_change":"2023-11-24T06:27:42.394418+0000","last_active":"2023-11-24T06:27:43.659266+0000","last_peered":"2023-11-24T06:27:43.659266+0000","last_clean":"2023-11-24T06:27:43.659266+0000","last_became_active":"2023-11-24T06:27:42.394103+0000","last_became_peered":"2023-11-24T06:27:42.394103+0000","last_unstale":"2023-11-24T06:27:43.659266+0000","last_undegraded":"2023-11-24T06:27:43.659266+0000","last_fullsized":"2023-11-24T06:27:43.659266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658516+0000","last_change":"2023-11-24T06:27:42.386091+0000","last_active":"2023-11-24T06:27:43.658516+0000","last_peered":"2023-11-24T06:27:43.658516+0000","last_clean":"2023-11-24T06:27:43.658516+0000","last_became_active":"2023-11-24T06:27:42.385638+0000","last_became_peered":"2023-11-24T06:27:42.385638+0000","last_unstale":"2023-11-24T06:27:43.658516+0000","last_undegraded":"2023-11-24T06:27:43.658516+0000","last_fullsized":"2023-11-24T06:27:43.658516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657666+0000","last_change":"2023-11-24T06:27:42.385847+0000","last_active":"2023-11-24T06:27:43.657666+0000","last_peered":"2023-11-24T06:27:43.657666+0000","last_clean":"2023-11-24T06:27:43.657666+0000","last_became_active":"2023-11-24T06:27:42.384548+0000","last_became_peered":"2023-11-24T06:27:42.384548+0000","last_unstale":"2023-11-24T06:27:43.657666+0000","last_undegraded":"2023-11-24T06:27:43.657666+0000","last_fullsized":"2023-11-24T06:27:43.657666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359608+0000","last_change":"2023-11-24T06:27:42.381135+0000","last_active":"2023-11-24T06:27:43.359608+0000","last_peered":"2023-11-24T06:27:43.359608+0000","last_clean":"2023-11-24T06:27:43.359608+0000","last_became_active":"2023-11-24T06:27:42.379951+0000","last_became_peered":"2023-11-24T06:27:42.379951+0000","last_unstale":"2023-11-24T06:27:43.359608+0000","last_undegraded":"2023-11-24T06:27:43.359608+0000","last_fullsized":"2023-11-24T06:27:43.359608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355537+0000","last_change":"2023-11-24T06:27:42.396365+0000","last_active":"2023-11-24T06:27:43.355537+0000","last_peered":"2023-11-24T06:27:43.355537+0000","last_clean":"2023-11-24T06:27:43.355537+0000","last_became_active":"2023-11-24T06:27:42.394255+0000","last_became_peered":"2023-11-24T06:27:42.394255+0000","last_unstale":"2023-11-24T06:27:43.355537+0000","last_undegraded":"2023-11-24T06:27:43.355537+0000","last_fullsized":"2023-11-24T06:27:43.355537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660061+0000","last_change":"2023-11-24T06:27:42.385401+0000","last_active":"2023-11-24T06:27:43.660061+0000","last_peered":"2023-11-24T06:27:43.660061+0000","last_clean":"2023-11-24T06:27:43.660061+0000","last_became_active":"2023-11-24T06:27:42.384001+0000","last_became_peered":"2023-11-24T06:27:42.384001+0000","last_unstale":"2023-11-24T06:27:43.660061+0000","last_undegraded":"2023-11-24T06:27:43.660061+0000","last_fullsized":"2023-11-24T06:27:43.660061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:50.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657883+0000","last_change":"2023-11-24T06:27:42.375262+0000","last_active":"2023-11-24T06:27:43.657883+0000","last_peered":"2023-11-24T06:27:43.657883+0000","last_clean":"2023-11-24T06:27:43.657883+0000","last_became_active":"2023-11-24T06:27:42.374748+0000","last_became_peered":"2023-11-24T06:27:42.374748+0000","last_unstale":"2023-11-24T06:27:43.657883+0000","last_undegraded":"2023-11-24T06:27:43.657883+0000","last_fullsized":"2023-11-24T06:27:43.657883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355370+0000","last_change":"2023-11-24T06:27:42.397612+0000","last_active":"2023-11-24T06:27:43.355370+0000","last_peered":"2023-11-24T06:27:43.355370+0000","last_clean":"2023-11-24T06:27:43.355370+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355370+0000","last_undegraded":"2023-11-24T06:27:43.355370+0000","last_fullsized":"2023-11-24T06:27:43.355370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359434+0000","last_change":"2023-11-24T06:27:42.388744+0000","last_active":"2023-11-24T06:27:43.359434+0000","last_peered":"2023-11-24T06:27:43.359434+0000","last_clean":"2023-11-24T06:27:43.359434+0000","last_became_active":"2023-11-24T06:27:42.388298+0000","last_became_peered":"2023-11-24T06:27:42.388298+0000","last_unstale":"2023-11-24T06:27:43.359434+0000","last_undegraded":"2023-11-24T06:27:43.359434+0000","last_fullsized":"2023-11-24T06:27:43.359434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658048+0000","last_change":"2023-11-24T06:27:42.377765+0000","last_active":"2023-11-24T06:27:43.658048+0000","last_peered":"2023-11-24T06:27:43.658048+0000","last_clean":"2023-11-24T06:27:43.658048+0000","last_became_active":"2023-11-24T06:27:42.377032+0000","last_became_peered":"2023-11-24T06:27:42.377032+0000","last_unstale":"2023-11-24T06:27:43.658048+0000","last_undegraded":"2023-11-24T06:27:43.658048+0000","last_fullsized":"2023-11-24T06:27:43.658048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658694+0000","last_change":"2023-11-24T06:27:42.390624+0000","last_active":"2023-11-24T06:27:43.658694+0000","last_peered":"2023-11-24T06:27:43.658694+0000","last_clean":"2023-11-24T06:27:43.658694+0000","last_became_active":"2023-11-24T06:27:42.387611+0000","last_became_peered":"2023-11-24T06:27:42.387611+0000","last_unstale":"2023-11-24T06:27:43.658694+0000","last_undegraded":"2023-11-24T06:27:43.658694+0000","last_fullsized":"2023-11-24T06:27:43.658694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658079+0000","last_change":"2023-11-24T06:27:42.384846+0000","last_active":"2023-11-24T06:27:43.658079+0000","last_peered":"2023-11-24T06:27:43.658079+0000","last_clean":"2023-11-24T06:27:43.658079+0000","last_became_active":"2023-11-24T06:27:42.384575+0000","last_became_peered":"2023-11-24T06:27:42.384575+0000","last_unstale":"2023-11-24T06:27:43.658079+0000","last_undegraded":"2023-11-24T06:27:43.658079+0000","last_fullsized":"2023-11-24T06:27:43.658079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660460+0000","last_change":"2023-11-24T06:27:42.393160+0000","last_active":"2023-11-24T06:27:43.660460+0000","last_peered":"2023-11-24T06:27:43.660460+0000","last_clean":"2023-11-24T06:27:43.660460+0000","last_became_active":"2023-11-24T06:27:42.392277+0000","last_became_peered":"2023-11-24T06:27:42.392277+0000","last_unstale":"2023-11-24T06:27:43.660460+0000","last_undegraded":"2023-11-24T06:27:43.660460+0000","last_fullsized":"2023-11-24T06:27:43.660460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355922+0000","last_change":"2023-11-24T06:27:42.390004+0000","last_active":"2023-11-24T06:27:43.355922+0000","last_peered":"2023-11-24T06:27:43.355922+0000","last_clean":"2023-11-24T06:27:43.355922+0000","last_became_active":"2023-11-24T06:27:42.389682+0000","last_became_peered":"2023-11-24T06:27:42.389682+0000","last_unstale":"2023-11-24T06:27:43.355922+0000","last_undegraded":"2023-11-24T06:27:43.355922+0000","last_fullsized":"2023-11-24T06:27:43.355922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355110+0000","last_change":"2023-11-24T06:27:42.380022+0000","last_active":"2023-11-24T06:27:43.355110+0000","last_peered":"2023-11-24T06:27:43.355110+0000","last_clean":"2023-11-24T06:27:43.355110+0000","last_became_active":"2023-11-24T06:27:42.379510+0000","last_became_peered":"2023-11-24T06:27:42.379510+0000","last_unstale":"2023-11-24T06:27:43.355110+0000","last_undegraded":"2023-11-24T06:27:43.355110+0000","last_fullsized":"2023-11-24T06:27:43.355110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355598+0000","last_change":"2023-11-24T06:27:42.392966+0000","last_active":"2023-11-24T06:27:43.355598+0000","last_peered":"2023-11-24T06:27:43.355598+0000","last_clean":"2023-11-24T06:27:43.355598+0000","last_became_active":"2023-11-24T06:27:42.392683+0000","last_became_peered":"2023-11-24T06:27:42.392683+0000","last_unstale":"2023-11-24T06:27:43.355598+0000","last_undegraded":"2023-11-24T06:27:43.355598+0000","last_fullsized":"2023-11-24T06:27:43.355598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356153+0000","last_change":"2023-11-24T06:27:42.384496+0000","last_active":"2023-11-24T06:27:43.356153+0000","last_peered":"2023-11-24T06:27:43.356153+0000","last_clean":"2023-11-24T06:27:43.356153+0000","last_became_active":"2023-11-24T06:27:42.384121+0000","last_became_peered":"2023-11-24T06:27:42.384121+0000","last_unstale":"2023-11-24T06:27:43.356153+0000","last_undegraded":"2023-11-24T06:27:43.356153+0000","last_fullsized":"2023-11-24T06:27:43.356153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:09.683680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658776+0000","last_change":"2023-11-24T06:27:42.386718+0000","last_active":"2023-11-24T06:27:43.658776+0000","last_peered":"2023-11-24T06:27:43.658776+0000","last_clean":"2023-11-24T06:27:43.658776+0000","last_became_active":"2023-11-24T06:27:42.386397+0000","last_became_peered":"2023-11-24T06:27:42.386397+0000","last_unstale":"2023-11-24T06:27:43.658776+0000","last_undegraded":"2023-11-24T06:27:43.658776+0000","last_fullsized":"2023-11-24T06:27:43.658776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658422+0000","last_change":"2023-11-24T06:27:42.389793+0000","last_active":"2023-11-24T06:27:43.658422+0000","last_peered":"2023-11-24T06:27:43.658422+0000","last_clean":"2023-11-24T06:27:43.658422+0000","last_became_active":"2023-11-24T06:27:42.389493+0000","last_became_peered":"2023-11-24T06:27:42.389493+0000","last_unstale":"2023-11-24T06:27:43.658422+0000","last_undegraded":"2023-11-24T06:27:43.658422+0000","last_fullsized":"2023-11-24T06:27:43.658422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356062+0000","last_change":"2023-11-24T06:27:42.389497+0000","last_active":"2023-11-24T06:27:43.356062+0000","last_peered":"2023-11-24T06:27:43.356062+0000","last_clean":"2023-11-24T06:27:43.356062+0000","last_became_active":"2023-11-24T06:27:42.389132+0000","last_became_peered":"2023-11-24T06:27:42.389132+0000","last_unstale":"2023-11-24T06:27:43.356062+0000","last_undegraded":"2023-11-24T06:27:43.356062+0000","last_fullsized":"2023-11-24T06:27:43.356062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:37.697060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660457+0000","last_change":"2023-11-24T06:27:42.398466+0000","last_active":"2023-11-24T06:27:43.660457+0000","last_peered":"2023-11-24T06:27:43.660457+0000","last_clean":"2023-11-24T06:27:43.660457+0000","last_became_active":"2023-11-24T06:27:42.397061+0000","last_became_peered":"2023-11-24T06:27:42.397061+0000","last_unstale":"2023-11-24T06:27:43.660457+0000","last_undegraded":"2023-11-24T06:27:43.660457+0000","last_fullsized":"2023-11-24T06:27:43.660457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356264+0000","last_change":"2023-11-24T06:27:42.385278+0000","last_active":"2023-11-24T06:27:43.356264+0000","last_peered":"2023-11-24T06:27:43.356264+0000","last_clean":"2023-11-24T06:27:43.356264+0000","last_became_active":"2023-11-24T06:27:42.384909+0000","last_became_peered":"2023-11-24T06:27:42.384909+0000","last_unstale":"2023-11-24T06:27:43.356264+0000","last_undegraded":"2023-11-24T06:27:43.356264+0000","last_fullsized":"2023-11-24T06:27:43.356264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656675+0000","last_change":"2023-11-24T06:27:42.382417+0000","last_active":"2023-11-24T06:27:43.656675+0000","last_peered":"2023-11-24T06:27:43.656675+0000","last_clean":"2023-11-24T06:27:43.656675+0000","last_became_active":"2023-11-24T06:27:42.381985+0000","last_became_peered":"2023-11-24T06:27:42.381985+0000","last_unstale":"2023-11-24T06:27:43.656675+0000","last_undegraded":"2023-11-24T06:27:43.656675+0000","last_fullsized":"2023-11-24T06:27:43.656675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656964+0000","last_change":"2023-11-24T06:27:42.395403+0000","last_active":"2023-11-24T06:27:43.656964+0000","last_peered":"2023-11-24T06:27:43.656964+0000","last_clean":"2023-11-24T06:27:43.656964+0000","last_became_active":"2023-11-24T06:27:42.394442+0000","last_became_peered":"2023-11-24T06:27:42.394442+0000","last_unstale":"2023-11-24T06:27:43.656964+0000","last_undegraded":"2023-11-24T06:27:43.656964+0000","last_fullsized":"2023-11-24T06:27:43.656964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357092+0000","last_change":"2023-11-24T06:27:42.375948+0000","last_active":"2023-11-24T06:27:43.357092+0000","last_peered":"2023-11-24T06:27:43.357092+0000","last_clean":"2023-11-24T06:27:43.357092+0000","last_became_active":"2023-11-24T06:27:42.374443+0000","last_became_peered":"2023-11-24T06:27:42.374443+0000","last_unstale":"2023-11-24T06:27:43.357092+0000","last_undegraded":"2023-11-24T06:27:43.357092+0000","last_fullsized":"2023-11-24T06:27:43.357092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:11.141034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355607+0000","last_change":"2023-11-24T06:27:41.381254+0000","last_active":"2023-11-24T06:27:43.355607+0000","last_peered":"2023-11-24T06:27:43.355607+0000","last_clean":"2023-11-24T06:27:43.355607+0000","last_became_active":"2023-11-24T06:27:41.380768+0000","last_became_peered":"2023-11-24T06:27:41.380768+0000","last_unstale":"2023-11-24T06:27:43.355607+0000","last_undegraded":"2023-11-24T06:27:43.355607+0000","last_fullsized":"2023-11-24T06:27:43.355607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660887+0000","last_change":"2023-11-24T06:27:42.393018+0000","last_active":"2023-11-24T06:27:43.660887+0000","last_peered":"2023-11-24T06:27:43.660887+0000","last_clean":"2023-11-24T06:27:43.660887+0000","last_became_active":"2023-11-24T06:27:42.391726+0000","last_became_peered":"2023-11-24T06:27:42.391726+0000","last_unstale":"2023-11-24T06:27:43.660887+0000","last_undegraded":"2023-11-24T06:27:43.660887+0000","last_fullsized":"2023-11-24T06:27:43.660887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657580+0000","last_change":"2023-11-24T06:27:41.383020+0000","last_active":"2023-11-24T06:27:43.657580+0000","last_peered":"2023-11-24T06:27:43.657580+0000","last_clean":"2023-11-24T06:27:43.657580+0000","last_became_active":"2023-11-24T06:27:41.382642+0000","last_became_peered":"2023-11-24T06:27:41.382642+0000","last_unstale":"2023-11-24T06:27:43.657580+0000","last_undegraded":"2023-11-24T06:27:43.657580+0000","last_fullsized":"2023-11-24T06:27:43.657580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:19.408882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355377+0000","last_change":"2023-11-24T06:27:41.382979+0000","last_active":"2023-11-24T06:27:43.355377+0000","last_peered":"2023-11-24T06:27:43.355377+0000","last_clean":"2023-11-24T06:27:43.355377+0000","last_became_active":"2023-11-24T06:27:41.381955+0000","last_became_peered":"2023-11-24T06:27:41.381955+0000","last_unstale":"2023-11-24T06:27:43.355377+0000","last_undegraded":"2023-11-24T06:27:43.355377+0000","last_fullsized":"2023-11-24T06:27:43.355377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.354996+0000","last_change":"2023-11-24T06:27:42.395234+0000","last_active":"2023-11-24T06:27:43.354996+0000","last_peered":"2023-11-24T06:27:43.354996+0000","last_clean":"2023-11-24T06:27:43.354996+0000","last_became_active":"2023-11-24T06:27:42.394890+0000","last_became_peered":"2023-11-24T06:27:42.394890+0000","last_unstale":"2023-11-24T06:27:43.354996+0000","last_undegraded":"2023-11-24T06:27:43.354996+0000","last_fullsized":"2023-11-24T06:27:43.354996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355626+0000","last_change":"2023-11-24T06:27:41.371070+0000","last_active":"2023-11-24T06:27:43.355626+0000","last_peered":"2023-11-24T06:27:43.355626+0000","last_clean":"2023-11-24T06:27:43.355626+0000","last_became_active":"2023-11-24T06:27:41.368037+0000","last_became_peered":"2023-11-24T06:27:41.368037+0000","last_unstale":"2023-11-24T06:27:43.355626+0000","last_undegraded":"2023-11-24T06:27:43.355626+0000","last_fullsized":"2023-11-24T06:27:43.355626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657673+0000","last_change":"2023-11-24T06:27:42.385239+0000","last_active":"2023-11-24T06:27:43.657673+0000","last_peered":"2023-11-24T06:27:43.657673+0000","last_clean":"2023-11-24T06:27:43.657673+0000","last_became_active":"2023-11-24T06:27:42.384436+0000","last_became_peered":"2023-11-24T06:27:42.384436+0000","last_unstale":"2023-11-24T06:27:43.657673+0000","last_undegraded":"2023-11-24T06:27:43.657673+0000","last_fullsized":"2023-11-24T06:27:43.657673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657925+0000","last_change":"2023-11-24T06:27:41.381169+0000","last_active":"2023-11-24T06:27:43.657925+0000","last_peered":"2023-11-24T06:27:43.657925+0000","last_clean":"2023-11-24T06:27:43.657925+0000","last_became_active":"2023-11-24T06:27:41.380835+0000","last_became_peered":"2023-11-24T06:27:41.380835+0000","last_unstale":"2023-11-24T06:27:43.657925+0000","last_undegraded":"2023-11-24T06:27:43.657925+0000","last_fullsized":"2023-11-24T06:27:43.657925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358731+0000","last_change":"2023-11-24T06:27:42.390319+0000","last_active":"2023-11-24T06:27:43.358731+0000","last_peered":"2023-11-24T06:27:43.358731+0000","last_clean":"2023-11-24T06:27:43.358731+0000","last_became_active":"2023-11-24T06:27:42.388441+0000","last_became_peered":"2023-11-24T06:27:42.388441+0000","last_unstale":"2023-11-24T06:27:43.358731+0000","last_undegraded":"2023-11-24T06:27:43.358731+0000","last_fullsized":"2023-11-24T06:27:43.358731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358844+0000","last_change":"2023-11-24T06:27:41.385916+0000","last_active":"2023-11-24T06:27:43.358844+0000","last_peered":"2023-11-24T06:27:43.358844+0000","last_clean":"2023-11-24T06:27:43.358844+0000","last_became_active":"2023-11-24T06:27:41.385609+0000","last_became_peered":"2023-11-24T06:27:41.385609+0000","last_unstale":"2023-11-24T06:27:43.358844+0000","last_undegraded":"2023-11-24T06:27:43.358844+0000","last_fullsized":"2023-11-24T06:27:43.358844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660892+0000","last_change":"2023-11-24T06:27:42.395957+0000","last_active":"2023-11-24T06:27:43.660892+0000","last_peered":"2023-11-24T06:27:43.660892+0000","last_clean":"2023-11-24T06:27:43.660892+0000","last_became_active":"2023-11-24T06:27:42.395222+0000","last_became_peered":"2023-11-24T06:27:42.395222+0000","last_unstale":"2023-11-24T06:27:43.660892+0000","last_undegraded":"2023-11-24T06:27:43.660892+0000","last_fullsized":"2023-11-24T06:27:43.660892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658670+0000","last_change":"2023-11-24T06:27:41.382629+0000","last_active":"2023-11-24T06:27:43.658670+0000","last_peered":"2023-11-24T06:27:43.658670+0000","last_clean":"2023-11-24T06:27:43.658670+0000","last_became_active":"2023-11-24T06:27:41.382323+0000","last_became_peered":"2023-11-24T06:27:41.382323+0000","last_unstale":"2023-11-24T06:27:43.658670+0000","last_undegraded":"2023-11-24T06:27:43.658670+0000","last_fullsized":"2023-11-24T06:27:43.658670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:38.362359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359838+0000","last_change":"2023-11-24T06:27:42.373431+0000","last_active":"2023-11-24T06:27:43.359838+0000","last_peered":"2023-11-24T06:27:43.359838+0000","last_clean":"2023-11-24T06:27:43.359838+0000","last_became_active":"2023-11-24T06:27:42.373106+0000","last_became_peered":"2023-11-24T06:27:42.373106+0000","last_unstale":"2023-11-24T06:27:43.359838+0000","last_undegraded":"2023-11-24T06:27:43.359838+0000","last_fullsized":"2023-11-24T06:27:43.359838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658482+0000","last_change":"2023-11-24T06:27:41.381556+0000","last_active":"2023-11-24T06:27:43.658482+0000","last_peered":"2023-11-24T06:27:43.658482+0000","last_clean":"2023-11-24T06:27:43.658482+0000","last_became_active":"2023-11-24T06:27:41.380552+0000","last_became_peered":"2023-11-24T06:27:41.380552+0000","last_unstale":"2023-11-24T06:27:43.658482+0000","last_undegraded":"2023-11-24T06:27:43.658482+0000","last_fullsized":"2023-11-24T06:27:43.658482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:48.968587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659001+0000","last_change":"2023-11-24T06:27:42.391222+0000","last_active":"2023-11-24T06:27:43.659001+0000","last_peered":"2023-11-24T06:27:43.659001+0000","last_clean":"2023-11-24T06:27:43.659001+0000","last_became_active":"2023-11-24T06:27:42.390910+0000","last_became_peered":"2023-11-24T06:27:42.390910+0000","last_unstale":"2023-11-24T06:27:43.659001+0000","last_undegraded":"2023-11-24T06:27:43.659001+0000","last_fullsized":"2023-11-24T06:27:43.659001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660728+0000","last_change":"2023-11-24T06:27:41.374697+0000","last_active":"2023-11-24T06:27:43.660728+0000","last_peered":"2023-11-24T06:27:43.660728+0000","last_clean":"2023-11-24T06:27:43.660728+0000","last_became_active":"2023-11-24T06:27:41.374419+0000","last_became_peered":"2023-11-24T06:27:41.374419+0000","last_unstale":"2023-11-24T06:27:43.660728+0000","last_undegraded":"2023-11-24T06:27:43.660728+0000","last_fullsized":"2023-11-24T06:27:43.660728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660079+0000","last_change":"2023-11-24T06:27:42.393174+0000","last_active":"2023-11-24T06:27:43.660079+0000","last_peered":"2023-11-24T06:27:43.660079+0000","last_clean":"2023-11-24T06:27:43.660079+0000","last_became_active":"2023-11-24T06:27:42.392964+0000","last_became_peered":"2023-11-24T06:27:42.392964+0000","last_unstale":"2023-11-24T06:27:43.660079+0000","last_undegraded":"2023-11-24T06:27:43.660079+0000","last_fullsized":"2023-11-24T06:27:43.660079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356229+0000","last_change":"2023-11-24T06:27:41.379781+0000","last_active":"2023-11-24T06:27:43.356229+0000","last_peered":"2023-11-24T06:27:43.356229+0000","last_clean":"2023-11-24T06:27:43.356229+0000","last_became_active":"2023-11-24T06:27:41.379484+0000","last_became_peered":"2023-11-24T06:27:41.379484+0000","last_unstale":"2023-11-24T06:27:43.356229+0000","last_undegraded":"2023-11-24T06:27:43.356229+0000","last_fullsized":"2023-11-24T06:27:43.356229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658712+0000","last_change":"2023-11-24T06:27:42.385192+0000","last_active":"2023-11-24T06:27:43.658712+0000","last_peered":"2023-11-24T06:27:43.658712+0000","last_clean":"2023-11-24T06:27:43.658712+0000","last_became_active":"2023-11-24T06:27:42.384777+0000","last_became_peered":"2023-11-24T06:27:42.384777+0000","last_unstale":"2023-11-24T06:27:43.658712+0000","last_undegraded":"2023-11-24T06:27:43.658712+0000","last_fullsized":"2023-11-24T06:27:43.658712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657173+0000","last_change":"2023-11-24T06:27:41.383717+0000","last_active":"2023-11-24T06:27:43.657173+0000","last_peered":"2023-11-24T06:27:43.657173+0000","last_clean":"2023-11-24T06:27:43.657173+0000","last_became_active":"2023-11-24T06:27:41.382246+0000","last_became_peered":"2023-11-24T06:27:41.382246+0000","last_unstale":"2023-11-24T06:27:43.657173+0000","last_undegraded":"2023-11-24T06:27:43.657173+0000","last_fullsized":"2023-11-24T06:27:43.657173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:04.357908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656917+0000","last_change":"2023-11-24T06:27:42.392505+0000","last_active":"2023-11-24T06:27:43.656917+0000","last_peered":"2023-11-24T06:27:43.656917+0000","last_clean":"2023-11-24T06:27:43.656917+0000","last_became_active":"2023-11-24T06:27:42.391756+0000","last_became_peered":"2023-11-24T06:27:42.391756+0000","last_unstale":"2023-11-24T06:27:43.656917+0000","last_undegraded":"2023-11-24T06:27:43.656917+0000","last_fullsized":"2023-11-24T06:27:43.656917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656383+0000","last_change":"2023-11-24T06:27:41.366927+0000","last_active":"2023-11-24T06:27:43.656383+0000","last_peered":"2023-11-24T06:27:43.656383+0000","last_clean":"2023-11-24T06:27:43.656383+0000","last_became_active":"2023-11-24T06:27:41.366457+0000","last_became_peered":"2023-11-24T06:27:41.366457+0000","last_unstale":"2023-11-24T06:27:43.656383+0000","last_undegraded":"2023-11-24T06:27:43.656383+0000","last_fullsized":"2023-11-24T06:27:43.656383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657400+0000","last_change":"2023-11-24T06:27:42.386516+0000","last_active":"2023-11-24T06:27:43.657400+0000","last_peered":"2023-11-24T06:27:43.657400+0000","last_clean":"2023-11-24T06:27:43.657400+0000","last_became_active":"2023-11-24T06:27:42.386236+0000","last_became_peered":"2023-11-24T06:27:42.386236+0000","last_unstale":"2023-11-24T06:27:43.657400+0000","last_undegraded":"2023-11-24T06:27:43.657400+0000","last_fullsized":"2023-11-24T06:27:43.657400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358955+0000","last_change":"2023-11-24T06:27:41.368887+0000","last_active":"2023-11-24T06:27:43.358955+0000","last_peered":"2023-11-24T06:27:43.358955+0000","last_clean":"2023-11-24T06:27:43.358955+0000","last_became_active":"2023-11-24T06:27:41.367135+0000","last_became_peered":"2023-11-24T06:27:41.367135+0000","last_unstale":"2023-11-24T06:27:43.358955+0000","last_undegraded":"2023-11-24T06:27:43.358955+0000","last_fullsized":"2023-11-24T06:27:43.358955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:11.584532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659288+0000","last_change":"2023-11-24T06:27:42.391675+0000","last_active":"2023-11-24T06:27:43.659288+0000","last_peered":"2023-11-24T06:27:43.659288+0000","last_clean":"2023-11-24T06:27:43.659288+0000","last_became_active":"2023-11-24T06:27:42.391212+0000","last_became_peered":"2023-11-24T06:27:42.391212+0000","last_unstale":"2023-11-24T06:27:43.659288+0000","last_undegraded":"2023-11-24T06:27:43.659288+0000","last_fullsized":"2023-11-24T06:27:43.659288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658283+0000","last_change":"2023-11-24T06:27:41.377474+0000","last_active":"2023-11-24T06:27:43.658283+0000","last_peered":"2023-11-24T06:27:43.658283+0000","last_clean":"2023-11-24T06:27:43.658283+0000","last_became_active":"2023-11-24T06:27:41.377079+0000","last_became_peered":"2023-11-24T06:27:41.377079+0000","last_unstale":"2023-11-24T06:27:43.658283+0000","last_undegraded":"2023-11-24T06:27:43.658283+0000","last_fullsized":"2023-11-24T06:27:43.658283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659627+0000","last_change":"2023-11-24T06:27:42.388823+0000","last_active":"2023-11-24T06:27:43.659627+0000","last_peered":"2023-11-24T06:27:43.659627+0000","last_clean":"2023-11-24T06:27:43.659627+0000","last_became_active":"2023-11-24T06:27:42.388562+0000","last_became_peered":"2023-11-24T06:27:42.388562+0000","last_unstale":"2023-11-24T06:27:43.659627+0000","last_undegraded":"2023-11-24T06:27:43.659627+0000","last_fullsized":"2023-11-24T06:27:43.659627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:02.353531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658740+0000","last_change":"2023-11-24T06:27:41.375328+0000","last_active":"2023-11-24T06:27:43.658740+0000","last_peered":"2023-11-24T06:27:43.658740+0000","last_clean":"2023-11-24T06:27:43.658740+0000","last_became_active":"2023-11-24T06:27:41.375049+0000","last_became_peered":"2023-11-24T06:27:41.375049+0000","last_unstale":"2023-11-24T06:27:43.658740+0000","last_undegraded":"2023-11-24T06:27:43.658740+0000","last_fullsized":"2023-11-24T06:27:43.658740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358153+0000","last_change":"2023-11-24T06:27:42.398402+0000","last_active":"2023-11-24T06:27:43.358153+0000","last_peered":"2023-11-24T06:27:43.358153+0000","last_clean":"2023-11-24T06:27:43.358153+0000","last_became_active":"2023-11-24T06:27:42.393706+0000","last_became_peered":"2023-11-24T06:27:42.393706+0000","last_unstale":"2023-11-24T06:27:43.358153+0000","last_undegraded":"2023-11-24T06:27:43.358153+0000","last_fullsized":"2023-11-24T06:27:43.358153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657227+0000","last_change":"2023-11-24T06:27:41.366299+0000","last_active":"2023-11-24T06:27:43.657227+0000","last_peered":"2023-11-24T06:27:43.657227+0000","last_clean":"2023-11-24T06:27:43.657227+0000","last_became_active":"2023-11-24T06:27:41.365987+0000","last_became_peered":"2023-11-24T06:27:41.365987+0000","last_unstale":"2023-11-24T06:27:43.657227+0000","last_undegraded":"2023-11-24T06:27:43.657227+0000","last_fullsized":"2023-11-24T06:27:43.657227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:31.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658289+0000","last_change":"2023-11-24T06:27:42.392311+0000","last_active":"2023-11-24T06:27:43.658289+0000","last_peered":"2023-11-24T06:27:43.658289+0000","last_clean":"2023-11-24T06:27:43.658289+0000","last_became_active":"2023-11-24T06:27:42.391517+0000","last_became_peered":"2023-11-24T06:27:42.391517+0000","last_unstale":"2023-11-24T06:27:43.658289+0000","last_undegraded":"2023-11-24T06:27:43.658289+0000","last_fullsized":"2023-11-24T06:27:43.658289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.372912+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:27:42.372912+0000","last_peered":"2023-11-24T06:27:42.372912+0000","last_clean":"2023-11-24T06:27:42.372912+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:27:42.372912+0000","last_undegraded":"2023-11-24T06:27:42.372912+0000","last_fullsized":"2023-11-24T06:27:42.372912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355438+0000","last_change":"2023-11-24T06:27:41.370135+0000","last_active":"2023-11-24T06:27:43.355438+0000","last_peered":"2023-11-24T06:27:43.355438+0000","last_clean":"2023-11-24T06:27:43.355438+0000","last_became_active":"2023-11-24T06:27:41.369372+0000","last_became_peered":"2023-11-24T06:27:41.369372+0000","last_unstale":"2023-11-24T06:27:43.355438+0000","last_undegraded":"2023-11-24T06:27:43.355438+0000","last_fullsized":"2023-11-24T06:27:43.355438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:07.305138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355221+0000","last_change":"2023-11-24T06:27:42.384732+0000","last_active":"2023-11-24T06:27:43.355221+0000","last_peered":"2023-11-24T06:27:43.355221+0000","last_clean":"2023-11-24T06:27:43.355221+0000","last_became_active":"2023-11-24T06:27:42.384083+0000","last_became_peered":"2023-11-24T06:27:42.384083+0000","last_unstale":"2023-11-24T06:27:43.355221+0000","last_undegraded":"2023-11-24T06:27:43.355221+0000","last_fullsized":"2023-11-24T06:27:43.355221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656745+0000","last_change":"2023-11-24T06:27:41.384707+0000","last_active":"2023-11-24T06:27:43.656745+0000","last_peered":"2023-11-24T06:27:43.656745+0000","last_clean":"2023-11-24T06:27:43.656745+0000","last_became_active":"2023-11-24T06:27:41.384397+0000","last_became_peered":"2023-11-24T06:27:41.384397+0000","last_unstale":"2023-11-24T06:27:43.656745+0000","last_undegraded":"2023-11-24T06:27:43.656745+0000","last_fullsized":"2023-11-24T06:27:43.656745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659143+0000","last_change":"2023-11-24T06:27:42.394007+0000","last_active":"2023-11-24T06:27:43.659143+0000","last_peered":"2023-11-24T06:27:43.659143+0000","last_clean":"2023-11-24T06:27:43.659143+0000","last_became_active":"2023-11-24T06:27:42.393636+0000","last_became_peered":"2023-11-24T06:27:42.393636+0000","last_unstale":"2023-11-24T06:27:43.659143+0000","last_undegraded":"2023-11-24T06:27:43.659143+0000","last_fullsized":"2023-11-24T06:27:43.659143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:05.800693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656767+0000","last_change":"2023-11-24T06:27:41.375368+0000","last_active":"2023-11-24T06:27:43.656767+0000","last_peered":"2023-11-24T06:27:43.656767+0000","last_clean":"2023-11-24T06:27:43.656767+0000","last_became_active":"2023-11-24T06:27:41.375062+0000","last_became_peered":"2023-11-24T06:27:41.375062+0000","last_unstale":"2023-11-24T06:27:43.656767+0000","last_undegraded":"2023-11-24T06:27:43.656767+0000","last_fullsized":"2023-11-24T06:27:43.656767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657273+0000","last_change":"2023-11-24T06:27:42.375816+0000","last_active":"2023-11-24T06:27:43.657273+0000","last_peered":"2023-11-24T06:27:43.657273+0000","last_clean":"2023-11-24T06:27:43.657273+0000","last_became_active":"2023-11-24T06:27:42.375209+0000","last_became_peered":"2023-11-24T06:27:42.375209+0000","last_unstale":"2023-11-24T06:27:43.657273+0000","last_undegraded":"2023-11-24T06:27:43.657273+0000","last_fullsized":"2023-11-24T06:27:43.657273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356462+0000","last_change":"2023-11-24T06:27:41.376512+0000","last_active":"2023-11-24T06:27:43.356462+0000","last_peered":"2023-11-24T06:27:43.356462+0000","last_clean":"2023-11-24T06:27:43.356462+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356462+0000","last_undegraded":"2023-11-24T06:27:43.356462+0000","last_fullsized":"2023-11-24T06:27:43.356462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358784+0000","last_change":"2023-11-24T06:27:42.395282+0000","last_active":"2023-11-24T06:27:43.358784+0000","last_peered":"2023-11-24T06:27:43.358784+0000","last_clean":"2023-11-24T06:27:43.358784+0000","last_became_active":"2023-11-24T06:27:42.394947+0000","last_became_peered":"2023-11-24T06:27:42.394947+0000","last_unstale":"2023-11-24T06:27:43.358784+0000","last_undegraded":"2023-11-24T06:27:43.358784+0000","last_fullsized":"2023-11-24T06:27:43.358784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:03.529167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658293+0000","last_change":"2023-11-24T06:27:41.377691+0000","last_active":"2023-11-24T06:27:43.658293+0000","last_peered":"2023-11-24T06:27:43.658293+0000","last_clean":"2023-11-24T06:27:43.658293+0000","last_became_active":"2023-11-24T06:27:41.377407+0000","last_became_peered":"2023-11-24T06:27:41.377407+0000","last_unstale":"2023-11-24T06:27:43.658293+0000","last_undegraded":"2023-11-24T06:27:43.658293+0000","last_fullsized":"2023-11-24T06:27:43.658293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:31.799834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358354+0000","last_change":"2023-11-24T06:27:42.388531+0000","last_active":"2023-11-24T06:27:43.358354+0000","last_peered":"2023-11-24T06:27:43.358354+0000","last_clean":"2023-11-24T06:27:43.358354+0000","last_became_active":"2023-11-24T06:27:42.388137+0000","last_became_peered":"2023-11-24T06:27:42.388137+0000","last_unstale":"2023-11-24T06:27:43.358354+0000","last_undegraded":"2023-11-24T06:27:43.358354+0000","last_fullsized":"2023-11-24T06:27:43.358354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658823+0000","last_change":"2023-11-24T06:27:41.383929+0000","last_active":"2023-11-24T06:27:43.658823+0000","last_peered":"2023-11-24T06:27:43.658823+0000","last_clean":"2023-11-24T06:27:43.658823+0000","last_became_active":"2023-11-24T06:27:41.383652+0000","last_became_peered":"2023-11-24T06:27:41.383652+0000","last_unstale":"2023-11-24T06:27:43.658823+0000","last_undegraded":"2023-11-24T06:27:43.658823+0000","last_fullsized":"2023-11-24T06:27:43.658823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658415+0000","last_change":"2023-11-24T06:27:42.378135+0000","last_active":"2023-11-24T06:27:43.658415+0000","last_peered":"2023-11-24T06:27:43.658415+0000","last_clean":"2023-11-24T06:27:43.658415+0000","last_became_active":"2023-11-24T06:27:42.377665+0000","last_became_peered":"2023-11-24T06:27:42.377665+0000","last_unstale":"2023-11-24T06:27:43.658415+0000","last_undegraded":"2023-11-24T06:27:43.658415+0000","last_fullsized":"2023-11-24T06:27:43.658415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:44.175851+0000","last_change":"2023-11-24T06:27:41.388698+0000","last_active":"2023-11-24T06:27:44.175851+0000","last_peered":"2023-11-24T06:27:44.175851+0000","last_clean":"2023-11-24T06:27:44.175851+0000","last_became_active":"2023-11-24T06:27:41.388379+0000","last_became_peered":"2023-11-24T06:27:41.388379+0000","last_unstale":"2023-11-24T06:27:44.175851+0000","last_undegraded":"2023-11-24T06:27:44.175851+0000","last_fullsized":"2023-11-24T06:27:44.175851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356256+0000","last_change":"2023-11-24T06:27:42.387047+0000","last_active":"2023-11-24T06:27:43.356256+0000","last_peered":"2023-11-24T06:27:43.356256+0000","last_clean":"2023-11-24T06:27:43.356256+0000","last_became_active":"2023-11-24T06:27:42.386729+0000","last_became_peered":"2023-11-24T06:27:42.386729+0000","last_unstale":"2023-11-24T06:27:43.356256+0000","last_undegraded":"2023-11-24T06:27:43.356256+0000","last_fullsized":"2023-11-24T06:27:43.356256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:31.853326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658188+0000","last_change":"2023-11-24T06:27:41.379713+0000","last_active":"2023-11-24T06:27:43.658188+0000","last_peered":"2023-11-24T06:27:43.658188+0000","last_clean":"2023-11-24T06:27:43.658188+0000","last_became_active":"2023-11-24T06:27:41.379416+0000","last_became_peered":"2023-11-24T06:27:41.379416+0000","last_unstale":"2023-11-24T06:27:43.658188+0000","last_undegraded":"2023-11-24T06:27:43.658188+0000","last_fullsized":"2023-11-24T06:27:43.658188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660748+0000","last_change":"2023-11-24T06:27:42.392358+0000","last_active":"2023-11-24T06:27:43.660748+0000","last_peered":"2023-11-24T06:27:43.660748+0000","last_clean":"2023-11-24T06:27:43.660748+0000","last_became_active":"2023-11-24T06:27:42.392029+0000","last_became_peered":"2023-11-24T06:27:42.392029+0000","last_unstale":"2023-11-24T06:27:43.660748+0000","last_undegraded":"2023-11-24T06:27:43.660748+0000","last_fullsized":"2023-11-24T06:27:43.660748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656706+0000","last_change":"2023-11-24T06:27:41.384873+0000","last_active":"2023-11-24T06:27:43.656706+0000","last_peered":"2023-11-24T06:27:43.656706+0000","last_clean":"2023-11-24T06:27:43.656706+0000","last_became_active":"2023-11-24T06:27:41.384475+0000","last_became_peered":"2023-11-24T06:27:41.384475+0000","last_unstale":"2023-11-24T06:27:43.656706+0000","last_undegraded":"2023-11-24T06:27:43.656706+0000","last_fullsized":"2023-11-24T06:27:43.656706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357526+0000","last_change":"2023-11-24T06:27:42.381195+0000","last_active":"2023-11-24T06:27:43.357526+0000","last_peered":"2023-11-24T06:27:43.357526+0000","last_clean":"2023-11-24T06:27:43.357526+0000","last_became_active":"2023-11-24T06:27:42.377313+0000","last_became_peered":"2023-11-24T06:27:42.377313+0000","last_unstale":"2023-11-24T06:27:43.357526+0000","last_undegraded":"2023-11-24T06:27:43.357526+0000","last_fullsized":"2023-11-24T06:27:43.357526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:32.562262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657763+0000","last_change":"2023-11-24T06:27:41.377860+0000","last_active":"2023-11-24T06:27:43.657763+0000","last_peered":"2023-11-24T06:27:43.657763+0000","last_clean":"2023-11-24T06:27:43.657763+0000","last_became_active":"2023-11-24T06:27:41.377572+0000","last_became_peered":"2023-11-24T06:27:41.377572+0000","last_unstale":"2023-11-24T06:27:43.657763+0000","last_undegraded":"2023-11-24T06:27:43.657763+0000","last_fullsized":"2023-11-24T06:27:43.657763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:52.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355182+0000","last_change":"2023-11-24T06:27:42.385894+0000","last_active":"2023-11-24T06:27:43.355182+0000","last_peered":"2023-11-24T06:27:43.355182+0000","last_clean":"2023-11-24T06:27:43.355182+0000","last_became_active":"2023-11-24T06:27:42.385586+0000","last_became_peered":"2023-11-24T06:27:42.385586+0000","last_unstale":"2023-11-24T06:27:43.355182+0000","last_undegraded":"2023-11-24T06:27:43.355182+0000","last_fullsized":"2023-11-24T06:27:43.355182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658185+0000","last_change":"2023-11-24T06:27:41.380746+0000","last_active":"2023-11-24T06:27:43.658185+0000","last_peered":"2023-11-24T06:27:43.658185+0000","last_clean":"2023-11-24T06:27:43.658185+0000","last_became_active":"2023-11-24T06:27:41.380535+0000","last_became_peered":"2023-11-24T06:27:41.380535+0000","last_unstale":"2023-11-24T06:27:43.658185+0000","last_undegraded":"2023-11-24T06:27:43.658185+0000","last_fullsized":"2023-11-24T06:27:43.658185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:10.313348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355700+0000","last_change":"2023-11-24T06:27:42.399877+0000","last_active":"2023-11-24T06:27:43.355700+0000","last_peered":"2023-11-24T06:27:43.355700+0000","last_clean":"2023-11-24T06:27:43.355700+0000","last_became_active":"2023-11-24T06:27:42.399051+0000","last_became_peered":"2023-11-24T06:27:42.399051+0000","last_unstale":"2023-11-24T06:27:43.355700+0000","last_undegraded":"2023-11-24T06:27:43.355700+0000","last_fullsized":"2023-11-24T06:27:43.355700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:09.167666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659116+0000","last_change":"2023-11-24T06:27:41.386935+0000","last_active":"2023-11-24T06:27:43.659116+0000","last_peered":"2023-11-24T06:27:43.659116+0000","last_clean":"2023-11-24T06:27:43.659116+0000","last_became_active":"2023-11-24T06:27:41.386607+0000","last_became_peered":"2023-11-24T06:27:41.386607+0000","last_unstale":"2023-11-24T06:27:43.659116+0000","last_undegraded":"2023-11-24T06:27:43.659116+0000","last_fullsized":"2023-11-24T06:27:43.659116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659499+0000","last_change":"2023-11-24T06:27:42.382231+0000","last_active":"2023-11-24T06:27:43.659499+0000","last_peered":"2023-11-24T06:27:43.659499+0000","last_clean":"2023-11-24T06:27:43.659499+0000","last_became_active":"2023-11-24T06:27:42.381452+0000","last_became_peered":"2023-11-24T06:27:42.381452+0000","last_unstale":"2023-11-24T06:27:43.659499+0000","last_undegraded":"2023-11-24T06:27:43.659499+0000","last_fullsized":"2023-11-24T06:27:43.659499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358206+0000","last_change":"2023-11-24T06:27:42.378166+0000","last_active":"2023-11-24T06:27:43.358206+0000","last_peered":"2023-11-24T06:27:43.358206+0000","last_clean":"2023-11-24T06:27:43.358206+0000","last_became_active":"2023-11-24T06:27:42.377859+0000","last_became_peered":"2023-11-24T06:27:42.377859+0000","last_unstale":"2023-11-24T06:27:43.358206+0000","last_undegraded":"2023-11-24T06:27:43.358206+0000","last_fullsized":"2023-11-24T06:27:43.358206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657894+0000","last_change":"2023-11-24T06:27:41.386057+0000","last_active":"2023-11-24T06:27:43.657894+0000","last_peered":"2023-11-24T06:27:43.657894+0000","last_clean":"2023-11-24T06:27:43.657894+0000","last_became_active":"2023-11-24T06:27:41.385754+0000","last_became_peered":"2023-11-24T06:27:41.385754+0000","last_unstale":"2023-11-24T06:27:43.657894+0000","last_undegraded":"2023-11-24T06:27:43.657894+0000","last_fullsized":"2023-11-24T06:27:43.657894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357781+0000","last_change":"2023-11-24T06:27:42.382235+0000","last_active":"2023-11-24T06:27:43.357781+0000","last_peered":"2023-11-24T06:27:43.357781+0000","last_clean":"2023-11-24T06:27:43.357781+0000","last_became_active":"2023-11-24T06:27:42.380559+0000","last_became_peered":"2023-11-24T06:27:42.380559+0000","last_unstale":"2023-11-24T06:27:43.357781+0000","last_undegraded":"2023-11-24T06:27:43.357781+0000","last_fullsized":"2023-11-24T06:27:43.357781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658434+0000","last_change":"2023-11-24T06:27:41.385695+0000","last_active":"2023-11-24T06:27:43.658434+0000","last_peered":"2023-11-24T06:27:43.658434+0000","last_clean":"2023-11-24T06:27:43.658434+0000","last_became_active":"2023-11-24T06:27:41.385158+0000","last_became_peered":"2023-11-24T06:27:41.385158+0000","last_unstale":"2023-11-24T06:27:43.658434+0000","last_undegraded":"2023-11-24T06:27:43.658434+0000","last_fullsized":"2023-11-24T06:27:43.658434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660564+0000","last_change":"2023-11-24T06:27:42.396117+0000","last_active":"2023-11-24T06:27:43.660564+0000","last_peered":"2023-11-24T06:27:43.660564+0000","last_clean":"2023-11-24T06:27:43.660564+0000","last_became_active":"2023-11-24T06:27:42.395511+0000","last_became_peered":"2023-11-24T06:27:42.395511+0000","last_unstale":"2023-11-24T06:27:43.660564+0000","last_undegraded":"2023-11-24T06:27:43.660564+0000","last_fullsized":"2023-11-24T06:27:43.660564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356285+0000","last_change":"2023-11-24T06:27:41.373126+0000","last_active":"2023-11-24T06:27:43.356285+0000","last_peered":"2023-11-24T06:27:43.356285+0000","last_clean":"2023-11-24T06:27:43.356285+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356285+0000","last_undegraded":"2023-11-24T06:27:43.356285+0000","last_fullsized":"2023-11-24T06:27:43.356285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:05.485619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356095+0000","last_change":"2023-11-24T06:27:42.386786+0000","last_active":"2023-11-24T06:27:43.356095+0000","last_peered":"2023-11-24T06:27:43.356095+0000","last_clean":"2023-11-24T06:27:43.356095+0000","last_became_active":"2023-11-24T06:27:42.386492+0000","last_became_peered":"2023-11-24T06:27:42.386492+0000","last_unstale":"2023-11-24T06:27:43.356095+0000","last_undegraded":"2023-11-24T06:27:43.356095+0000","last_fullsized":"2023-11-24T06:27:43.356095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:36:11.379388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.661047+0000","last_change":"2023-11-24T06:27:41.378175+0000","last_active":"2023-11-24T06:27:43.661047+0000","last_peered":"2023-11-24T06:27:43.661047+0000","last_clean":"2023-11-24T06:27:43.661047+0000","last_became_active":"2023-11-24T06:27:41.376774+0000","last_became_peered":"2023-11-24T06:27:41.376774+0000","last_unstale":"2023-11-24T06:27:43.661047+0000","last_undegraded":"2023-11-24T06:27:43.661047+0000","last_fullsized":"2023-11-24T06:27:43.661047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":823296,"data_stored":669752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.323}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3849999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.542}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0550000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2029999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3119999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1779999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8460000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:52.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:53.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.203+0000 7f8e4f0da640 1 Processor -- start 2023-11-24T06:27:53.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.203+0000 7f8e4f0da640 1 -- start start 2023-11-24T06:27:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4f0da640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e481449e0 0x7f8e480cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4f0da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4806ea40 0x7f8e4815b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4f0da640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e4815bab0 0x7f8e4806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4f0da640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e48144dc0 con 0x7f8e481449e0 2023-11-24T06:27:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4f0da640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4806f9c0 con 0x7f8e4815bab0 2023-11-24T06:27:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4f0da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4806ee80 con 0x7f8e4806ea40 2023-11-24T06:27:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4e0d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e481449e0 0x7f8e480cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4e0d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e481449e0 0x7f8e480cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41876/0 (socket says 172.21.15.64:41876) 2023-11-24T06:27:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.205+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/2286496240 learned_addr learned my addr 172.21.15.64:0/2286496240 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4d8d7640 1 --2- 172.21.15.64:0/2286496240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4806ea40 0x7f8e4815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4e8d9640 1 --2- 172.21.15.64:0/2286496240 >> v2:172.21.15.162:3300/0 conn(0x7f8e4815bab0 0x7f8e4806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/2286496240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4806ea40 msgr2=0x7f8e4815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4e0d8640 1 --2- 172.21.15.64:0/2286496240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4806ea40 0x7f8e4815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/2286496240 >> v2:172.21.15.162:3300/0 conn(0x7f8e4815bab0 msgr2=0x7f8e4806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4e0d8640 1 --2- 172.21.15.64:0/2286496240 >> v2:172.21.15.162:3300/0 conn(0x7f8e4815bab0 0x7f8e4806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/2286496240 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4806f120 con 0x7f8e481449e0 2023-11-24T06:27:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.206+0000 7f8e4e8d9640 1 --2- 172.21.15.64:0/2286496240 >> v2:172.21.15.162:3300/0 conn(0x7f8e4815bab0 0x7f8e4806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:53.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.208+0000 7f8e4e0d8640 1 --2- 172.21.15.64:0/2286496240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e481449e0 0x7f8e480cc920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e480b5470 tx=0x7f8e4402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d52b0bfea5a094a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.208+0000 7f8e4d0d6640 1 -- 172.21.15.64:0/2286496240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440317e0 con 0x7f8e481449e0 2023-11-24T06:27:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.208+0000 7f8e4d0d6640 1 -- 172.21.15.64:0/2286496240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44031940 con 0x7f8e481449e0 2023-11-24T06:27:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.208+0000 7f8e4d0d6640 1 -- 172.21.15.64:0/2286496240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44031cd0 con 0x7f8e481449e0 2023-11-24T06:27:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 -- 172.21.15.64:0/2286496240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e481449e0 msgr2=0x7f8e480cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/2286496240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e481449e0 0x7f8e480cc920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e480b5470 tx=0x7f8e4402b880 comp rx=0 tx=0).stop 2023-11-24T06:27:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 -- 172.21.15.64:0/2286496240 shutdown_connections 2023-11-24T06:27:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/2286496240 >> v2:172.21.15.162:3300/0 conn(0x7f8e4815bab0 0x7f8e4806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/2286496240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4806ea40 0x7f8e4815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/2286496240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e481449e0 0x7f8e480cc920 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 -- 172.21.15.64:0/2286496240 >> 172.21.15.64:0/2286496240 conn(0x7f8e480b1cd0 msgr2=0x7f8e480b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 -- 172.21.15.64:0/2286496240 shutdown_connections 2023-11-24T06:27:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.209+0000 7f8e4f0da640 1 -- 172.21.15.64:0/2286496240 wait complete. 2023-11-24T06:27:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.211+0000 7f8e4f0da640 1 Processor -- start 2023-11-24T06:27:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.211+0000 7f8e4f0da640 1 -- start start 2023-11-24T06:27:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4f0da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4806ea40 0x7f8e481437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4f0da640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e481449e0 0x7f8e4815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4f0da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4815bab0 0x7f8e480aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4f0da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48143ce0 con 0x7f8e4806ea40 2023-11-24T06:27:53.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4f0da640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4815af80 con 0x7f8e481449e0 2023-11-24T06:27:53.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4f0da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48143310 con 0x7f8e4815bab0 2023-11-24T06:27:53.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4e0d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4806ea40 0x7f8e481437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4e0d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4806ea40 0x7f8e481437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41890/0 (socket says 172.21.15.64:41890) 2023-11-24T06:27:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.212+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/1458937971 learned_addr learned my addr 172.21.15.64:0/1458937971 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4e8d9640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4815bab0 0x7f8e480aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4d8d7640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e481449e0 0x7f8e4815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4815bab0 msgr2=0x7f8e480aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4e0d8640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4815bab0 0x7f8e480aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e481449e0 msgr2=0x7f8e4815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4e0d8640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e481449e0 0x7f8e4815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4e0d8640 1 -- 172.21.15.64:0/1458937971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440094d0 con 0x7f8e4806ea40 2023-11-24T06:27:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4e0d8640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4806ea40 0x7f8e481437a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e44000c00 tx=0x7f8e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e3effd640 1 -- 172.21.15.64:0/1458937971 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44032880 con 0x7f8e4806ea40 2023-11-24T06:27:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e3effd640 1 -- 172.21.15.64:0/1458937971 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44032a00 con 0x7f8e4806ea40 2023-11-24T06:27:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e481429e0 con 0x7f8e4806ea40 2023-11-24T06:27:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.213+0000 7f8e3effd640 1 -- 172.21.15.64:0/1458937971 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44032db0 con 0x7f8e4806ea40 2023-11-24T06:27:53.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.214+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e48142fa0 con 0x7f8e4806ea40 2023-11-24T06:27:53.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.215+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4806ad00 con 0x7f8e4806ea40 2023-11-24T06:27:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.216+0000 7f8e3effd640 1 -- 172.21.15.64:0/1458937971 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e440453f0 con 0x7f8e4806ea40 2023-11-24T06:27:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.216+0000 7f8e3effd640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8e2c06ae40 0x7f8e2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.216+0000 7f8e3effd640 1 -- 172.21.15.64:0/1458937971 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e440ab880 con 0x7f8e4806ea40 2023-11-24T06:27:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.217+0000 7f8e4d8d7640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8e2c06ae40 0x7f8e2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.217+0000 7f8e4d8d7640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8e2c06ae40 0x7f8e2c06d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e481f6240 tx=0x7f8e38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.221+0000 7f8e3effd640 1 -- 172.21.15.64:0/1458937971 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e44076890 con 0x7f8e4806ea40 2023-11-24T06:27:53.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.407+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e4806d6d0 con 0x7f8e2c06ae40 2023-11-24T06:27:53.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.416+0000 7f8e3effd640 1 -- 172.21.15.64:0/1458937971 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264351 (secure 0 0 0) 0x7f8e3800f830 con 0x7f8e2c06ae40 2023-11-24T06:27:53.421 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:53.424 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:27:53.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8e2c06ae40 msgr2=0x7f8e2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8e2c06ae40 0x7f8e2c06d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e481f6240 tx=0x7f8e38008040 comp rx=0 tx=0).stop 2023-11-24T06:27:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4806ea40 msgr2=0x7f8e481437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4806ea40 0x7f8e481437a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e44000c00 tx=0x7f8e440046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 shutdown_connections 2023-11-24T06:27:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f8e2c06ae40 0x7f8e2c06d300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4815bab0 0x7f8e480aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e481449e0 0x7f8e4815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 --2- 172.21.15.64:0/1458937971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4806ea40 0x7f8e481437a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 >> 172.21.15.64:0/1458937971 conn(0x7f8e480b1cd0 msgr2=0x7f8e4815b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.422+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 shutdown_connections 2023-11-24T06:27:53.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.423+0000 7f8e4f0da640 1 -- 172.21.15.64:0/1458937971 wait complete. 2023-11-24T06:27:53.445 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-24T06:27:53.105093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219328,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755392,"statfs":{"total":773094113280,"available":772869521408,"internally_reserved":0,"allocated":3055616,"data_stored":1815972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004032"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658921+0000","last_change":"2023-11-24T06:27:42.394622+0000","last_active":"2023-11-24T06:27:43.658921+0000","last_peered":"2023-11-24T06:27:43.658921+0000","last_clean":"2023-11-24T06:27:43.658921+0000","last_became_active":"2023-11-24T06:27:42.394278+0000","last_became_peered":"2023-11-24T06:27:42.394278+0000","last_unstale":"2023-11-24T06:27:43.658921+0000","last_undegraded":"2023-11-24T06:27:43.658921+0000","last_fullsized":"2023-11-24T06:27:43.658921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659042+0000","last_change":"2023-11-24T06:27:42.395699+0000","last_active":"2023-11-24T06:27:43.659042+0000","last_peered":"2023-11-24T06:27:43.659042+0000","last_clean":"2023-11-24T06:27:43.659042+0000","last_became_active":"2023-11-24T06:27:42.395424+0000","last_became_peered":"2023-11-24T06:27:42.395424+0000","last_unstale":"2023-11-24T06:27:43.659042+0000","last_undegraded":"2023-11-24T06:27:43.659042+0000","last_fullsized":"2023-11-24T06:27:43.659042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660634+0000","last_change":"2023-11-24T06:27:42.390431+0000","last_active":"2023-11-24T06:27:43.660634+0000","last_peered":"2023-11-24T06:27:43.660634+0000","last_clean":"2023-11-24T06:27:43.660634+0000","last_became_active":"2023-11-24T06:27:42.390163+0000","last_became_peered":"2023-11-24T06:27:42.390163+0000","last_unstale":"2023-11-24T06:27:43.660634+0000","last_undegraded":"2023-11-24T06:27:43.660634+0000","last_fullsized":"2023-11-24T06:27:43.660634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357785+0000","last_change":"2023-11-24T06:27:42.393940+0000","last_active":"2023-11-24T06:27:43.357785+0000","last_peered":"2023-11-24T06:27:43.357785+0000","last_clean":"2023-11-24T06:27:43.357785+0000","last_became_active":"2023-11-24T06:27:42.393341+0000","last_became_peered":"2023-11-24T06:27:42.393341+0000","last_unstale":"2023-11-24T06:27:43.357785+0000","last_undegraded":"2023-11-24T06:27:43.357785+0000","last_fullsized":"2023-11-24T06:27:43.357785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658343+0000","last_change":"2023-11-24T06:27:42.392760+0000","last_active":"2023-11-24T06:27:43.658343+0000","last_peered":"2023-11-24T06:27:43.658343+0000","last_clean":"2023-11-24T06:27:43.658343+0000","last_became_active":"2023-11-24T06:27:42.390011+0000","last_became_peered":"2023-11-24T06:27:42.390011+0000","last_unstale":"2023-11-24T06:27:43.658343+0000","last_undegraded":"2023-11-24T06:27:43.658343+0000","last_fullsized":"2023-11-24T06:27:43.658343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:20.681063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355889+0000","last_change":"2023-11-24T06:27:42.397895+0000","last_active":"2023-11-24T06:27:43.355889+0000","last_peered":"2023-11-24T06:27:43.355889+0000","last_clean":"2023-11-24T06:27:43.355889+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355889+0000","last_undegraded":"2023-11-24T06:27:43.355889+0000","last_fullsized":"2023-11-24T06:27:43.355889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657379+0000","last_change":"2023-11-24T06:27:42.398329+0000","last_active":"2023-11-24T06:27:43.657379+0000","last_peered":"2023-11-24T06:27:43.657379+0000","last_clean":"2023-11-24T06:27:43.657379+0000","last_became_active":"2023-11-24T06:27:42.398095+0000","last_became_peered":"2023-11-24T06:27:42.398095+0000","last_unstale":"2023-11-24T06:27:43.657379+0000","last_undegraded":"2023-11-24T06:27:43.657379+0000","last_fullsized":"2023-11-24T06:27:43.657379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355033+0000","last_change":"2023-11-24T06:27:42.395940+0000","last_active":"2023-11-24T06:27:43.355033+0000","last_peered":"2023-11-24T06:27:43.355033+0000","last_clean":"2023-11-24T06:27:43.355033+0000","last_became_active":"2023-11-24T06:27:42.395595+0000","last_became_peered":"2023-11-24T06:27:42.395595+0000","last_unstale":"2023-11-24T06:27:43.355033+0000","last_undegraded":"2023-11-24T06:27:43.355033+0000","last_fullsized":"2023-11-24T06:27:43.355033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659266+0000","last_change":"2023-11-24T06:27:42.394418+0000","last_active":"2023-11-24T06:27:43.659266+0000","last_peered":"2023-11-24T06:27:43.659266+0000","last_clean":"2023-11-24T06:27:43.659266+0000","last_became_active":"2023-11-24T06:27:42.394103+0000","last_became_peered":"2023-11-24T06:27:42.394103+0000","last_unstale":"2023-11-24T06:27:43.659266+0000","last_undegraded":"2023-11-24T06:27:43.659266+0000","last_fullsized":"2023-11-24T06:27:43.659266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658516+0000","last_change":"2023-11-24T06:27:42.386091+0000","last_active":"2023-11-24T06:27:43.658516+0000","last_peered":"2023-11-24T06:27:43.658516+0000","last_clean":"2023-11-24T06:27:43.658516+0000","last_became_active":"2023-11-24T06:27:42.385638+0000","last_became_peered":"2023-11-24T06:27:42.385638+0000","last_unstale":"2023-11-24T06:27:43.658516+0000","last_undegraded":"2023-11-24T06:27:43.658516+0000","last_fullsized":"2023-11-24T06:27:43.658516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657666+0000","last_change":"2023-11-24T06:27:42.385847+0000","last_active":"2023-11-24T06:27:43.657666+0000","last_peered":"2023-11-24T06:27:43.657666+0000","last_clean":"2023-11-24T06:27:43.657666+0000","last_became_active":"2023-11-24T06:27:42.384548+0000","last_became_peered":"2023-11-24T06:27:42.384548+0000","last_unstale":"2023-11-24T06:27:43.657666+0000","last_undegraded":"2023-11-24T06:27:43.657666+0000","last_fullsized":"2023-11-24T06:27:43.657666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359608+0000","last_change":"2023-11-24T06:27:42.381135+0000","last_active":"2023-11-24T06:27:43.359608+0000","last_peered":"2023-11-24T06:27:43.359608+0000","last_clean":"2023-11-24T06:27:43.359608+0000","last_became_active":"2023-11-24T06:27:42.379951+0000","last_became_peered":"2023-11-24T06:27:42.379951+0000","last_unstale":"2023-11-24T06:27:43.359608+0000","last_undegraded":"2023-11-24T06:27:43.359608+0000","last_fullsized":"2023-11-24T06:27:43.359608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355537+0000","last_change":"2023-11-24T06:27:42.396365+0000","last_active":"2023-11-24T06:27:43.355537+0000","last_peered":"2023-11-24T06:27:43.355537+0000","last_clean":"2023-11-24T06:27:43.355537+0000","last_became_active":"2023-11-24T06:27:42.394255+0000","last_became_peered":"2023-11-24T06:27:42.394255+0000","last_unstale":"2023-11-24T06:27:43.355537+0000","last_undegraded":"2023-11-24T06:27:43.355537+0000","last_fullsized":"2023-11-24T06:27:43.355537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:10:19.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660061+0000","last_change":"2023-11-24T06:27:42.385401+0000","last_active":"2023-11-24T06:27:43.660061+0000","last_peered":"2023-11-24T06:27:43.660061+0000","last_clean":"2023-11-24T06:27:43.660061+0000","last_became_active":"2023-11-24T06:27:42.384001+0000","last_became_peered":"2023-11-24T06:27:42.384001+0000","last_unstale":"2023-11-24T06:27:43.660061+0000","last_undegraded":"2023-11-24T06:27:43.660061+0000","last_fullsized":"2023-11-24T06:27:43.660061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:50.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657883+0000","last_change":"2023-11-24T06:27:42.375262+0000","last_active":"2023-11-24T06:27:43.657883+0000","last_peered":"2023-11-24T06:27:43.657883+0000","last_clean":"2023-11-24T06:27:43.657883+0000","last_became_active":"2023-11-24T06:27:42.374748+0000","last_became_peered":"2023-11-24T06:27:42.374748+0000","last_unstale":"2023-11-24T06:27:43.657883+0000","last_undegraded":"2023-11-24T06:27:43.657883+0000","last_fullsized":"2023-11-24T06:27:43.657883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355370+0000","last_change":"2023-11-24T06:27:42.397612+0000","last_active":"2023-11-24T06:27:43.355370+0000","last_peered":"2023-11-24T06:27:43.355370+0000","last_clean":"2023-11-24T06:27:43.355370+0000","last_became_active":"2023-11-24T06:27:42.397291+0000","last_became_peered":"2023-11-24T06:27:42.397291+0000","last_unstale":"2023-11-24T06:27:43.355370+0000","last_undegraded":"2023-11-24T06:27:43.355370+0000","last_fullsized":"2023-11-24T06:27:43.355370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:46.201131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359434+0000","last_change":"2023-11-24T06:27:42.388744+0000","last_active":"2023-11-24T06:27:43.359434+0000","last_peered":"2023-11-24T06:27:43.359434+0000","last_clean":"2023-11-24T06:27:43.359434+0000","last_became_active":"2023-11-24T06:27:42.388298+0000","last_became_peered":"2023-11-24T06:27:42.388298+0000","last_unstale":"2023-11-24T06:27:43.359434+0000","last_undegraded":"2023-11-24T06:27:43.359434+0000","last_fullsized":"2023-11-24T06:27:43.359434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658048+0000","last_change":"2023-11-24T06:27:42.377765+0000","last_active":"2023-11-24T06:27:43.658048+0000","last_peered":"2023-11-24T06:27:43.658048+0000","last_clean":"2023-11-24T06:27:43.658048+0000","last_became_active":"2023-11-24T06:27:42.377032+0000","last_became_peered":"2023-11-24T06:27:42.377032+0000","last_unstale":"2023-11-24T06:27:43.658048+0000","last_undegraded":"2023-11-24T06:27:43.658048+0000","last_fullsized":"2023-11-24T06:27:43.658048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658694+0000","last_change":"2023-11-24T06:27:42.390624+0000","last_active":"2023-11-24T06:27:43.658694+0000","last_peered":"2023-11-24T06:27:43.658694+0000","last_clean":"2023-11-24T06:27:43.658694+0000","last_became_active":"2023-11-24T06:27:42.387611+0000","last_became_peered":"2023-11-24T06:27:42.387611+0000","last_unstale":"2023-11-24T06:27:43.658694+0000","last_undegraded":"2023-11-24T06:27:43.658694+0000","last_fullsized":"2023-11-24T06:27:43.658694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658079+0000","last_change":"2023-11-24T06:27:42.384846+0000","last_active":"2023-11-24T06:27:43.658079+0000","last_peered":"2023-11-24T06:27:43.658079+0000","last_clean":"2023-11-24T06:27:43.658079+0000","last_became_active":"2023-11-24T06:27:42.384575+0000","last_became_peered":"2023-11-24T06:27:42.384575+0000","last_unstale":"2023-11-24T06:27:43.658079+0000","last_undegraded":"2023-11-24T06:27:43.658079+0000","last_fullsized":"2023-11-24T06:27:43.658079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660460+0000","last_change":"2023-11-24T06:27:42.393160+0000","last_active":"2023-11-24T06:27:43.660460+0000","last_peered":"2023-11-24T06:27:43.660460+0000","last_clean":"2023-11-24T06:27:43.660460+0000","last_became_active":"2023-11-24T06:27:42.392277+0000","last_became_peered":"2023-11-24T06:27:42.392277+0000","last_unstale":"2023-11-24T06:27:43.660460+0000","last_undegraded":"2023-11-24T06:27:43.660460+0000","last_fullsized":"2023-11-24T06:27:43.660460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:49.487595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355922+0000","last_change":"2023-11-24T06:27:42.390004+0000","last_active":"2023-11-24T06:27:43.355922+0000","last_peered":"2023-11-24T06:27:43.355922+0000","last_clean":"2023-11-24T06:27:43.355922+0000","last_became_active":"2023-11-24T06:27:42.389682+0000","last_became_peered":"2023-11-24T06:27:42.389682+0000","last_unstale":"2023-11-24T06:27:43.355922+0000","last_undegraded":"2023-11-24T06:27:43.355922+0000","last_fullsized":"2023-11-24T06:27:43.355922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355110+0000","last_change":"2023-11-24T06:27:42.380022+0000","last_active":"2023-11-24T06:27:43.355110+0000","last_peered":"2023-11-24T06:27:43.355110+0000","last_clean":"2023-11-24T06:27:43.355110+0000","last_became_active":"2023-11-24T06:27:42.379510+0000","last_became_peered":"2023-11-24T06:27:42.379510+0000","last_unstale":"2023-11-24T06:27:43.355110+0000","last_undegraded":"2023-11-24T06:27:43.355110+0000","last_fullsized":"2023-11-24T06:27:43.355110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:00.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355598+0000","last_change":"2023-11-24T06:27:42.392966+0000","last_active":"2023-11-24T06:27:43.355598+0000","last_peered":"2023-11-24T06:27:43.355598+0000","last_clean":"2023-11-24T06:27:43.355598+0000","last_became_active":"2023-11-24T06:27:42.392683+0000","last_became_peered":"2023-11-24T06:27:42.392683+0000","last_unstale":"2023-11-24T06:27:43.355598+0000","last_undegraded":"2023-11-24T06:27:43.355598+0000","last_fullsized":"2023-11-24T06:27:43.355598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356153+0000","last_change":"2023-11-24T06:27:42.384496+0000","last_active":"2023-11-24T06:27:43.356153+0000","last_peered":"2023-11-24T06:27:43.356153+0000","last_clean":"2023-11-24T06:27:43.356153+0000","last_became_active":"2023-11-24T06:27:42.384121+0000","last_became_peered":"2023-11-24T06:27:42.384121+0000","last_unstale":"2023-11-24T06:27:43.356153+0000","last_undegraded":"2023-11-24T06:27:43.356153+0000","last_fullsized":"2023-11-24T06:27:43.356153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:09.683680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658776+0000","last_change":"2023-11-24T06:27:42.386718+0000","last_active":"2023-11-24T06:27:43.658776+0000","last_peered":"2023-11-24T06:27:43.658776+0000","last_clean":"2023-11-24T06:27:43.658776+0000","last_became_active":"2023-11-24T06:27:42.386397+0000","last_became_peered":"2023-11-24T06:27:42.386397+0000","last_unstale":"2023-11-24T06:27:43.658776+0000","last_undegraded":"2023-11-24T06:27:43.658776+0000","last_fullsized":"2023-11-24T06:27:43.658776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658422+0000","last_change":"2023-11-24T06:27:42.389793+0000","last_active":"2023-11-24T06:27:43.658422+0000","last_peered":"2023-11-24T06:27:43.658422+0000","last_clean":"2023-11-24T06:27:43.658422+0000","last_became_active":"2023-11-24T06:27:42.389493+0000","last_became_peered":"2023-11-24T06:27:42.389493+0000","last_unstale":"2023-11-24T06:27:43.658422+0000","last_undegraded":"2023-11-24T06:27:43.658422+0000","last_fullsized":"2023-11-24T06:27:43.658422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356062+0000","last_change":"2023-11-24T06:27:42.389497+0000","last_active":"2023-11-24T06:27:43.356062+0000","last_peered":"2023-11-24T06:27:43.356062+0000","last_clean":"2023-11-24T06:27:43.356062+0000","last_became_active":"2023-11-24T06:27:42.389132+0000","last_became_peered":"2023-11-24T06:27:42.389132+0000","last_unstale":"2023-11-24T06:27:43.356062+0000","last_undegraded":"2023-11-24T06:27:43.356062+0000","last_fullsized":"2023-11-24T06:27:43.356062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:37.697060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660457+0000","last_change":"2023-11-24T06:27:42.398466+0000","last_active":"2023-11-24T06:27:43.660457+0000","last_peered":"2023-11-24T06:27:43.660457+0000","last_clean":"2023-11-24T06:27:43.660457+0000","last_became_active":"2023-11-24T06:27:42.397061+0000","last_became_peered":"2023-11-24T06:27:42.397061+0000","last_unstale":"2023-11-24T06:27:43.660457+0000","last_undegraded":"2023-11-24T06:27:43.660457+0000","last_fullsized":"2023-11-24T06:27:43.660457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356264+0000","last_change":"2023-11-24T06:27:42.385278+0000","last_active":"2023-11-24T06:27:43.356264+0000","last_peered":"2023-11-24T06:27:43.356264+0000","last_clean":"2023-11-24T06:27:43.356264+0000","last_became_active":"2023-11-24T06:27:42.384909+0000","last_became_peered":"2023-11-24T06:27:42.384909+0000","last_unstale":"2023-11-24T06:27:43.356264+0000","last_undegraded":"2023-11-24T06:27:43.356264+0000","last_fullsized":"2023-11-24T06:27:43.356264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656675+0000","last_change":"2023-11-24T06:27:42.382417+0000","last_active":"2023-11-24T06:27:43.656675+0000","last_peered":"2023-11-24T06:27:43.656675+0000","last_clean":"2023-11-24T06:27:43.656675+0000","last_became_active":"2023-11-24T06:27:42.381985+0000","last_became_peered":"2023-11-24T06:27:42.381985+0000","last_unstale":"2023-11-24T06:27:43.656675+0000","last_undegraded":"2023-11-24T06:27:43.656675+0000","last_fullsized":"2023-11-24T06:27:43.656675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:02.346285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656964+0000","last_change":"2023-11-24T06:27:42.395403+0000","last_active":"2023-11-24T06:27:43.656964+0000","last_peered":"2023-11-24T06:27:43.656964+0000","last_clean":"2023-11-24T06:27:43.656964+0000","last_became_active":"2023-11-24T06:27:42.394442+0000","last_became_peered":"2023-11-24T06:27:42.394442+0000","last_unstale":"2023-11-24T06:27:43.656964+0000","last_undegraded":"2023-11-24T06:27:43.656964+0000","last_fullsized":"2023-11-24T06:27:43.656964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:43.477705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357092+0000","last_change":"2023-11-24T06:27:42.375948+0000","last_active":"2023-11-24T06:27:43.357092+0000","last_peered":"2023-11-24T06:27:43.357092+0000","last_clean":"2023-11-24T06:27:43.357092+0000","last_became_active":"2023-11-24T06:27:42.374443+0000","last_became_peered":"2023-11-24T06:27:42.374443+0000","last_unstale":"2023-11-24T06:27:43.357092+0000","last_undegraded":"2023-11-24T06:27:43.357092+0000","last_fullsized":"2023-11-24T06:27:43.357092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:11.141034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355607+0000","last_change":"2023-11-24T06:27:41.381254+0000","last_active":"2023-11-24T06:27:43.355607+0000","last_peered":"2023-11-24T06:27:43.355607+0000","last_clean":"2023-11-24T06:27:43.355607+0000","last_became_active":"2023-11-24T06:27:41.380768+0000","last_became_peered":"2023-11-24T06:27:41.380768+0000","last_unstale":"2023-11-24T06:27:43.355607+0000","last_undegraded":"2023-11-24T06:27:43.355607+0000","last_fullsized":"2023-11-24T06:27:43.355607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660887+0000","last_change":"2023-11-24T06:27:42.393018+0000","last_active":"2023-11-24T06:27:43.660887+0000","last_peered":"2023-11-24T06:27:43.660887+0000","last_clean":"2023-11-24T06:27:43.660887+0000","last_became_active":"2023-11-24T06:27:42.391726+0000","last_became_peered":"2023-11-24T06:27:42.391726+0000","last_unstale":"2023-11-24T06:27:43.660887+0000","last_undegraded":"2023-11-24T06:27:43.660887+0000","last_fullsized":"2023-11-24T06:27:43.660887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657580+0000","last_change":"2023-11-24T06:27:41.383020+0000","last_active":"2023-11-24T06:27:43.657580+0000","last_peered":"2023-11-24T06:27:43.657580+0000","last_clean":"2023-11-24T06:27:43.657580+0000","last_became_active":"2023-11-24T06:27:41.382642+0000","last_became_peered":"2023-11-24T06:27:41.382642+0000","last_unstale":"2023-11-24T06:27:43.657580+0000","last_undegraded":"2023-11-24T06:27:43.657580+0000","last_fullsized":"2023-11-24T06:27:43.657580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:19.408882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355377+0000","last_change":"2023-11-24T06:27:41.382979+0000","last_active":"2023-11-24T06:27:43.355377+0000","last_peered":"2023-11-24T06:27:43.355377+0000","last_clean":"2023-11-24T06:27:43.355377+0000","last_became_active":"2023-11-24T06:27:41.381955+0000","last_became_peered":"2023-11-24T06:27:41.381955+0000","last_unstale":"2023-11-24T06:27:43.355377+0000","last_undegraded":"2023-11-24T06:27:43.355377+0000","last_fullsized":"2023-11-24T06:27:43.355377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.354996+0000","last_change":"2023-11-24T06:27:42.395234+0000","last_active":"2023-11-24T06:27:43.354996+0000","last_peered":"2023-11-24T06:27:43.354996+0000","last_clean":"2023-11-24T06:27:43.354996+0000","last_became_active":"2023-11-24T06:27:42.394890+0000","last_became_peered":"2023-11-24T06:27:42.394890+0000","last_unstale":"2023-11-24T06:27:43.354996+0000","last_undegraded":"2023-11-24T06:27:43.354996+0000","last_fullsized":"2023-11-24T06:27:43.354996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355626+0000","last_change":"2023-11-24T06:27:41.371070+0000","last_active":"2023-11-24T06:27:43.355626+0000","last_peered":"2023-11-24T06:27:43.355626+0000","last_clean":"2023-11-24T06:27:43.355626+0000","last_became_active":"2023-11-24T06:27:41.368037+0000","last_became_peered":"2023-11-24T06:27:41.368037+0000","last_unstale":"2023-11-24T06:27:43.355626+0000","last_undegraded":"2023-11-24T06:27:43.355626+0000","last_fullsized":"2023-11-24T06:27:43.355626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657673+0000","last_change":"2023-11-24T06:27:42.385239+0000","last_active":"2023-11-24T06:27:43.657673+0000","last_peered":"2023-11-24T06:27:43.657673+0000","last_clean":"2023-11-24T06:27:43.657673+0000","last_became_active":"2023-11-24T06:27:42.384436+0000","last_became_peered":"2023-11-24T06:27:42.384436+0000","last_unstale":"2023-11-24T06:27:43.657673+0000","last_undegraded":"2023-11-24T06:27:43.657673+0000","last_fullsized":"2023-11-24T06:27:43.657673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:13.626447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657925+0000","last_change":"2023-11-24T06:27:41.381169+0000","last_active":"2023-11-24T06:27:43.657925+0000","last_peered":"2023-11-24T06:27:43.657925+0000","last_clean":"2023-11-24T06:27:43.657925+0000","last_became_active":"2023-11-24T06:27:41.380835+0000","last_became_peered":"2023-11-24T06:27:41.380835+0000","last_unstale":"2023-11-24T06:27:43.657925+0000","last_undegraded":"2023-11-24T06:27:43.657925+0000","last_fullsized":"2023-11-24T06:27:43.657925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358731+0000","last_change":"2023-11-24T06:27:42.390319+0000","last_active":"2023-11-24T06:27:43.358731+0000","last_peered":"2023-11-24T06:27:43.358731+0000","last_clean":"2023-11-24T06:27:43.358731+0000","last_became_active":"2023-11-24T06:27:42.388441+0000","last_became_peered":"2023-11-24T06:27:42.388441+0000","last_unstale":"2023-11-24T06:27:43.358731+0000","last_undegraded":"2023-11-24T06:27:43.358731+0000","last_fullsized":"2023-11-24T06:27:43.358731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.920436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358844+0000","last_change":"2023-11-24T06:27:41.385916+0000","last_active":"2023-11-24T06:27:43.358844+0000","last_peered":"2023-11-24T06:27:43.358844+0000","last_clean":"2023-11-24T06:27:43.358844+0000","last_became_active":"2023-11-24T06:27:41.385609+0000","last_became_peered":"2023-11-24T06:27:41.385609+0000","last_unstale":"2023-11-24T06:27:43.358844+0000","last_undegraded":"2023-11-24T06:27:43.358844+0000","last_fullsized":"2023-11-24T06:27:43.358844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660892+0000","last_change":"2023-11-24T06:27:42.395957+0000","last_active":"2023-11-24T06:27:43.660892+0000","last_peered":"2023-11-24T06:27:43.660892+0000","last_clean":"2023-11-24T06:27:43.660892+0000","last_became_active":"2023-11-24T06:27:42.395222+0000","last_became_peered":"2023-11-24T06:27:42.395222+0000","last_unstale":"2023-11-24T06:27:43.660892+0000","last_undegraded":"2023-11-24T06:27:43.660892+0000","last_fullsized":"2023-11-24T06:27:43.660892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658670+0000","last_change":"2023-11-24T06:27:41.382629+0000","last_active":"2023-11-24T06:27:43.658670+0000","last_peered":"2023-11-24T06:27:43.658670+0000","last_clean":"2023-11-24T06:27:43.658670+0000","last_became_active":"2023-11-24T06:27:41.382323+0000","last_became_peered":"2023-11-24T06:27:41.382323+0000","last_unstale":"2023-11-24T06:27:43.658670+0000","last_undegraded":"2023-11-24T06:27:43.658670+0000","last_fullsized":"2023-11-24T06:27:43.658670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:38.362359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.359838+0000","last_change":"2023-11-24T06:27:42.373431+0000","last_active":"2023-11-24T06:27:43.359838+0000","last_peered":"2023-11-24T06:27:43.359838+0000","last_clean":"2023-11-24T06:27:43.359838+0000","last_became_active":"2023-11-24T06:27:42.373106+0000","last_became_peered":"2023-11-24T06:27:42.373106+0000","last_unstale":"2023-11-24T06:27:43.359838+0000","last_undegraded":"2023-11-24T06:27:43.359838+0000","last_fullsized":"2023-11-24T06:27:43.359838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:20.851667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658482+0000","last_change":"2023-11-24T06:27:41.381556+0000","last_active":"2023-11-24T06:27:43.658482+0000","last_peered":"2023-11-24T06:27:43.658482+0000","last_clean":"2023-11-24T06:27:43.658482+0000","last_became_active":"2023-11-24T06:27:41.380552+0000","last_became_peered":"2023-11-24T06:27:41.380552+0000","last_unstale":"2023-11-24T06:27:43.658482+0000","last_undegraded":"2023-11-24T06:27:43.658482+0000","last_fullsized":"2023-11-24T06:27:43.658482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:48.968587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659001+0000","last_change":"2023-11-24T06:27:42.391222+0000","last_active":"2023-11-24T06:27:43.659001+0000","last_peered":"2023-11-24T06:27:43.659001+0000","last_clean":"2023-11-24T06:27:43.659001+0000","last_became_active":"2023-11-24T06:27:42.390910+0000","last_became_peered":"2023-11-24T06:27:42.390910+0000","last_unstale":"2023-11-24T06:27:43.659001+0000","last_undegraded":"2023-11-24T06:27:43.659001+0000","last_fullsized":"2023-11-24T06:27:43.659001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660728+0000","last_change":"2023-11-24T06:27:41.374697+0000","last_active":"2023-11-24T06:27:43.660728+0000","last_peered":"2023-11-24T06:27:43.660728+0000","last_clean":"2023-11-24T06:27:43.660728+0000","last_became_active":"2023-11-24T06:27:41.374419+0000","last_became_peered":"2023-11-24T06:27:41.374419+0000","last_unstale":"2023-11-24T06:27:43.660728+0000","last_undegraded":"2023-11-24T06:27:43.660728+0000","last_fullsized":"2023-11-24T06:27:43.660728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660079+0000","last_change":"2023-11-24T06:27:42.393174+0000","last_active":"2023-11-24T06:27:43.660079+0000","last_peered":"2023-11-24T06:27:43.660079+0000","last_clean":"2023-11-24T06:27:43.660079+0000","last_became_active":"2023-11-24T06:27:42.392964+0000","last_became_peered":"2023-11-24T06:27:42.392964+0000","last_unstale":"2023-11-24T06:27:43.660079+0000","last_undegraded":"2023-11-24T06:27:43.660079+0000","last_fullsized":"2023-11-24T06:27:43.660079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356229+0000","last_change":"2023-11-24T06:27:41.379781+0000","last_active":"2023-11-24T06:27:43.356229+0000","last_peered":"2023-11-24T06:27:43.356229+0000","last_clean":"2023-11-24T06:27:43.356229+0000","last_became_active":"2023-11-24T06:27:41.379484+0000","last_became_peered":"2023-11-24T06:27:41.379484+0000","last_unstale":"2023-11-24T06:27:43.356229+0000","last_undegraded":"2023-11-24T06:27:43.356229+0000","last_fullsized":"2023-11-24T06:27:43.356229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658712+0000","last_change":"2023-11-24T06:27:42.385192+0000","last_active":"2023-11-24T06:27:43.658712+0000","last_peered":"2023-11-24T06:27:43.658712+0000","last_clean":"2023-11-24T06:27:43.658712+0000","last_became_active":"2023-11-24T06:27:42.384777+0000","last_became_peered":"2023-11-24T06:27:42.384777+0000","last_unstale":"2023-11-24T06:27:43.658712+0000","last_undegraded":"2023-11-24T06:27:43.658712+0000","last_fullsized":"2023-11-24T06:27:43.658712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657173+0000","last_change":"2023-11-24T06:27:41.383717+0000","last_active":"2023-11-24T06:27:43.657173+0000","last_peered":"2023-11-24T06:27:43.657173+0000","last_clean":"2023-11-24T06:27:43.657173+0000","last_became_active":"2023-11-24T06:27:41.382246+0000","last_became_peered":"2023-11-24T06:27:41.382246+0000","last_unstale":"2023-11-24T06:27:43.657173+0000","last_undegraded":"2023-11-24T06:27:43.657173+0000","last_fullsized":"2023-11-24T06:27:43.657173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:04.357908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656917+0000","last_change":"2023-11-24T06:27:42.392505+0000","last_active":"2023-11-24T06:27:43.656917+0000","last_peered":"2023-11-24T06:27:43.656917+0000","last_clean":"2023-11-24T06:27:43.656917+0000","last_became_active":"2023-11-24T06:27:42.391756+0000","last_became_peered":"2023-11-24T06:27:42.391756+0000","last_unstale":"2023-11-24T06:27:43.656917+0000","last_undegraded":"2023-11-24T06:27:43.656917+0000","last_fullsized":"2023-11-24T06:27:43.656917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.164921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656383+0000","last_change":"2023-11-24T06:27:41.366927+0000","last_active":"2023-11-24T06:27:43.656383+0000","last_peered":"2023-11-24T06:27:43.656383+0000","last_clean":"2023-11-24T06:27:43.656383+0000","last_became_active":"2023-11-24T06:27:41.366457+0000","last_became_peered":"2023-11-24T06:27:41.366457+0000","last_unstale":"2023-11-24T06:27:43.656383+0000","last_undegraded":"2023-11-24T06:27:43.656383+0000","last_fullsized":"2023-11-24T06:27:43.656383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657400+0000","last_change":"2023-11-24T06:27:42.386516+0000","last_active":"2023-11-24T06:27:43.657400+0000","last_peered":"2023-11-24T06:27:43.657400+0000","last_clean":"2023-11-24T06:27:43.657400+0000","last_became_active":"2023-11-24T06:27:42.386236+0000","last_became_peered":"2023-11-24T06:27:42.386236+0000","last_unstale":"2023-11-24T06:27:43.657400+0000","last_undegraded":"2023-11-24T06:27:43.657400+0000","last_fullsized":"2023-11-24T06:27:43.657400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:25:30.995937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358955+0000","last_change":"2023-11-24T06:27:41.368887+0000","last_active":"2023-11-24T06:27:43.358955+0000","last_peered":"2023-11-24T06:27:43.358955+0000","last_clean":"2023-11-24T06:27:43.358955+0000","last_became_active":"2023-11-24T06:27:41.367135+0000","last_became_peered":"2023-11-24T06:27:41.367135+0000","last_unstale":"2023-11-24T06:27:43.358955+0000","last_undegraded":"2023-11-24T06:27:43.358955+0000","last_fullsized":"2023-11-24T06:27:43.358955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:11.584532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659288+0000","last_change":"2023-11-24T06:27:42.391675+0000","last_active":"2023-11-24T06:27:43.659288+0000","last_peered":"2023-11-24T06:27:43.659288+0000","last_clean":"2023-11-24T06:27:43.659288+0000","last_became_active":"2023-11-24T06:27:42.391212+0000","last_became_peered":"2023-11-24T06:27:42.391212+0000","last_unstale":"2023-11-24T06:27:43.659288+0000","last_undegraded":"2023-11-24T06:27:43.659288+0000","last_fullsized":"2023-11-24T06:27:43.659288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:05:54.967404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658283+0000","last_change":"2023-11-24T06:27:41.377474+0000","last_active":"2023-11-24T06:27:43.658283+0000","last_peered":"2023-11-24T06:27:43.658283+0000","last_clean":"2023-11-24T06:27:43.658283+0000","last_became_active":"2023-11-24T06:27:41.377079+0000","last_became_peered":"2023-11-24T06:27:41.377079+0000","last_unstale":"2023-11-24T06:27:43.658283+0000","last_undegraded":"2023-11-24T06:27:43.658283+0000","last_fullsized":"2023-11-24T06:27:43.658283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659627+0000","last_change":"2023-11-24T06:27:42.388823+0000","last_active":"2023-11-24T06:27:43.659627+0000","last_peered":"2023-11-24T06:27:43.659627+0000","last_clean":"2023-11-24T06:27:43.659627+0000","last_became_active":"2023-11-24T06:27:42.388562+0000","last_became_peered":"2023-11-24T06:27:42.388562+0000","last_unstale":"2023-11-24T06:27:43.659627+0000","last_undegraded":"2023-11-24T06:27:43.659627+0000","last_fullsized":"2023-11-24T06:27:43.659627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:02.353531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658740+0000","last_change":"2023-11-24T06:27:41.375328+0000","last_active":"2023-11-24T06:27:43.658740+0000","last_peered":"2023-11-24T06:27:43.658740+0000","last_clean":"2023-11-24T06:27:43.658740+0000","last_became_active":"2023-11-24T06:27:41.375049+0000","last_became_peered":"2023-11-24T06:27:41.375049+0000","last_unstale":"2023-11-24T06:27:43.658740+0000","last_undegraded":"2023-11-24T06:27:43.658740+0000","last_fullsized":"2023-11-24T06:27:43.658740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:38.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358153+0000","last_change":"2023-11-24T06:27:42.398402+0000","last_active":"2023-11-24T06:27:43.358153+0000","last_peered":"2023-11-24T06:27:43.358153+0000","last_clean":"2023-11-24T06:27:43.358153+0000","last_became_active":"2023-11-24T06:27:42.393706+0000","last_became_peered":"2023-11-24T06:27:42.393706+0000","last_unstale":"2023-11-24T06:27:43.358153+0000","last_undegraded":"2023-11-24T06:27:43.358153+0000","last_fullsized":"2023-11-24T06:27:43.358153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:17.152877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657227+0000","last_change":"2023-11-24T06:27:41.366299+0000","last_active":"2023-11-24T06:27:43.657227+0000","last_peered":"2023-11-24T06:27:43.657227+0000","last_clean":"2023-11-24T06:27:43.657227+0000","last_became_active":"2023-11-24T06:27:41.365987+0000","last_became_peered":"2023-11-24T06:27:41.365987+0000","last_unstale":"2023-11-24T06:27:43.657227+0000","last_undegraded":"2023-11-24T06:27:43.657227+0000","last_fullsized":"2023-11-24T06:27:43.657227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:31.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658289+0000","last_change":"2023-11-24T06:27:42.392311+0000","last_active":"2023-11-24T06:27:43.658289+0000","last_peered":"2023-11-24T06:27:43.658289+0000","last_clean":"2023-11-24T06:27:43.658289+0000","last_became_active":"2023-11-24T06:27:42.391517+0000","last_became_peered":"2023-11-24T06:27:42.391517+0000","last_unstale":"2023-11-24T06:27:43.658289+0000","last_undegraded":"2023-11-24T06:27:43.658289+0000","last_fullsized":"2023-11-24T06:27:43.658289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:20.394422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.372912+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:27:42.372912+0000","last_peered":"2023-11-24T06:27:42.372912+0000","last_clean":"2023-11-24T06:27:42.372912+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:27:42.372912+0000","last_undegraded":"2023-11-24T06:27:42.372912+0000","last_fullsized":"2023-11-24T06:27:42.372912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355438+0000","last_change":"2023-11-24T06:27:41.370135+0000","last_active":"2023-11-24T06:27:43.355438+0000","last_peered":"2023-11-24T06:27:43.355438+0000","last_clean":"2023-11-24T06:27:43.355438+0000","last_became_active":"2023-11-24T06:27:41.369372+0000","last_became_peered":"2023-11-24T06:27:41.369372+0000","last_unstale":"2023-11-24T06:27:43.355438+0000","last_undegraded":"2023-11-24T06:27:43.355438+0000","last_fullsized":"2023-11-24T06:27:43.355438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:07.305138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355221+0000","last_change":"2023-11-24T06:27:42.384732+0000","last_active":"2023-11-24T06:27:43.355221+0000","last_peered":"2023-11-24T06:27:43.355221+0000","last_clean":"2023-11-24T06:27:43.355221+0000","last_became_active":"2023-11-24T06:27:42.384083+0000","last_became_peered":"2023-11-24T06:27:42.384083+0000","last_unstale":"2023-11-24T06:27:43.355221+0000","last_undegraded":"2023-11-24T06:27:43.355221+0000","last_fullsized":"2023-11-24T06:27:43.355221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:21.006563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656745+0000","last_change":"2023-11-24T06:27:41.384707+0000","last_active":"2023-11-24T06:27:43.656745+0000","last_peered":"2023-11-24T06:27:43.656745+0000","last_clean":"2023-11-24T06:27:43.656745+0000","last_became_active":"2023-11-24T06:27:41.384397+0000","last_became_peered":"2023-11-24T06:27:41.384397+0000","last_unstale":"2023-11-24T06:27:43.656745+0000","last_undegraded":"2023-11-24T06:27:43.656745+0000","last_fullsized":"2023-11-24T06:27:43.656745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:09.698249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659143+0000","last_change":"2023-11-24T06:27:42.394007+0000","last_active":"2023-11-24T06:27:43.659143+0000","last_peered":"2023-11-24T06:27:43.659143+0000","last_clean":"2023-11-24T06:27:43.659143+0000","last_became_active":"2023-11-24T06:27:42.393636+0000","last_became_peered":"2023-11-24T06:27:42.393636+0000","last_unstale":"2023-11-24T06:27:43.659143+0000","last_undegraded":"2023-11-24T06:27:43.659143+0000","last_fullsized":"2023-11-24T06:27:43.659143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:05.800693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656767+0000","last_change":"2023-11-24T06:27:41.375368+0000","last_active":"2023-11-24T06:27:43.656767+0000","last_peered":"2023-11-24T06:27:43.656767+0000","last_clean":"2023-11-24T06:27:43.656767+0000","last_became_active":"2023-11-24T06:27:41.375062+0000","last_became_peered":"2023-11-24T06:27:41.375062+0000","last_unstale":"2023-11-24T06:27:43.656767+0000","last_undegraded":"2023-11-24T06:27:43.656767+0000","last_fullsized":"2023-11-24T06:27:43.656767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:02.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657273+0000","last_change":"2023-11-24T06:27:42.375816+0000","last_active":"2023-11-24T06:27:43.657273+0000","last_peered":"2023-11-24T06:27:43.657273+0000","last_clean":"2023-11-24T06:27:43.657273+0000","last_became_active":"2023-11-24T06:27:42.375209+0000","last_became_peered":"2023-11-24T06:27:42.375209+0000","last_unstale":"2023-11-24T06:27:43.657273+0000","last_undegraded":"2023-11-24T06:27:43.657273+0000","last_fullsized":"2023-11-24T06:27:43.657273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:31.476314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356462+0000","last_change":"2023-11-24T06:27:41.376512+0000","last_active":"2023-11-24T06:27:43.356462+0000","last_peered":"2023-11-24T06:27:43.356462+0000","last_clean":"2023-11-24T06:27:43.356462+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356462+0000","last_undegraded":"2023-11-24T06:27:43.356462+0000","last_fullsized":"2023-11-24T06:27:43.356462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358784+0000","last_change":"2023-11-24T06:27:42.395282+0000","last_active":"2023-11-24T06:27:43.358784+0000","last_peered":"2023-11-24T06:27:43.358784+0000","last_clean":"2023-11-24T06:27:43.358784+0000","last_became_active":"2023-11-24T06:27:42.394947+0000","last_became_peered":"2023-11-24T06:27:42.394947+0000","last_unstale":"2023-11-24T06:27:43.358784+0000","last_undegraded":"2023-11-24T06:27:43.358784+0000","last_fullsized":"2023-11-24T06:27:43.358784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:03.529167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658293+0000","last_change":"2023-11-24T06:27:41.377691+0000","last_active":"2023-11-24T06:27:43.658293+0000","last_peered":"2023-11-24T06:27:43.658293+0000","last_clean":"2023-11-24T06:27:43.658293+0000","last_became_active":"2023-11-24T06:27:41.377407+0000","last_became_peered":"2023-11-24T06:27:41.377407+0000","last_unstale":"2023-11-24T06:27:43.658293+0000","last_undegraded":"2023-11-24T06:27:43.658293+0000","last_fullsized":"2023-11-24T06:27:43.658293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:31.799834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358354+0000","last_change":"2023-11-24T06:27:42.388531+0000","last_active":"2023-11-24T06:27:43.358354+0000","last_peered":"2023-11-24T06:27:43.358354+0000","last_clean":"2023-11-24T06:27:43.358354+0000","last_became_active":"2023-11-24T06:27:42.388137+0000","last_became_peered":"2023-11-24T06:27:42.388137+0000","last_unstale":"2023-11-24T06:27:43.358354+0000","last_undegraded":"2023-11-24T06:27:43.358354+0000","last_fullsized":"2023-11-24T06:27:43.358354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:07.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658823+0000","last_change":"2023-11-24T06:27:41.383929+0000","last_active":"2023-11-24T06:27:43.658823+0000","last_peered":"2023-11-24T06:27:43.658823+0000","last_clean":"2023-11-24T06:27:43.658823+0000","last_became_active":"2023-11-24T06:27:41.383652+0000","last_became_peered":"2023-11-24T06:27:41.383652+0000","last_unstale":"2023-11-24T06:27:43.658823+0000","last_undegraded":"2023-11-24T06:27:43.658823+0000","last_fullsized":"2023-11-24T06:27:43.658823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:29:59.931529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658415+0000","last_change":"2023-11-24T06:27:42.378135+0000","last_active":"2023-11-24T06:27:43.658415+0000","last_peered":"2023-11-24T06:27:43.658415+0000","last_clean":"2023-11-24T06:27:43.658415+0000","last_became_active":"2023-11-24T06:27:42.377665+0000","last_became_peered":"2023-11-24T06:27:42.377665+0000","last_unstale":"2023-11-24T06:27:43.658415+0000","last_undegraded":"2023-11-24T06:27:43.658415+0000","last_fullsized":"2023-11-24T06:27:43.658415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:44.175851+0000","last_change":"2023-11-24T06:27:41.388698+0000","last_active":"2023-11-24T06:27:44.175851+0000","last_peered":"2023-11-24T06:27:44.175851+0000","last_clean":"2023-11-24T06:27:44.175851+0000","last_became_active":"2023-11-24T06:27:41.388379+0000","last_became_peered":"2023-11-24T06:27:41.388379+0000","last_unstale":"2023-11-24T06:27:44.175851+0000","last_undegraded":"2023-11-24T06:27:44.175851+0000","last_fullsized":"2023-11-24T06:27:44.175851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:28.163406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356256+0000","last_change":"2023-11-24T06:27:42.387047+0000","last_active":"2023-11-24T06:27:43.356256+0000","last_peered":"2023-11-24T06:27:43.356256+0000","last_clean":"2023-11-24T06:27:43.356256+0000","last_became_active":"2023-11-24T06:27:42.386729+0000","last_became_peered":"2023-11-24T06:27:42.386729+0000","last_unstale":"2023-11-24T06:27:43.356256+0000","last_undegraded":"2023-11-24T06:27:43.356256+0000","last_fullsized":"2023-11-24T06:27:43.356256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:31.853326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658188+0000","last_change":"2023-11-24T06:27:41.379713+0000","last_active":"2023-11-24T06:27:43.658188+0000","last_peered":"2023-11-24T06:27:43.658188+0000","last_clean":"2023-11-24T06:27:43.658188+0000","last_became_active":"2023-11-24T06:27:41.379416+0000","last_became_peered":"2023-11-24T06:27:41.379416+0000","last_unstale":"2023-11-24T06:27:43.658188+0000","last_undegraded":"2023-11-24T06:27:43.658188+0000","last_fullsized":"2023-11-24T06:27:43.658188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660748+0000","last_change":"2023-11-24T06:27:42.392358+0000","last_active":"2023-11-24T06:27:43.660748+0000","last_peered":"2023-11-24T06:27:43.660748+0000","last_clean":"2023-11-24T06:27:43.660748+0000","last_became_active":"2023-11-24T06:27:42.392029+0000","last_became_peered":"2023-11-24T06:27:42.392029+0000","last_unstale":"2023-11-24T06:27:43.660748+0000","last_undegraded":"2023-11-24T06:27:43.660748+0000","last_fullsized":"2023-11-24T06:27:43.660748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:05.816271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.656706+0000","last_change":"2023-11-24T06:27:41.384873+0000","last_active":"2023-11-24T06:27:43.656706+0000","last_peered":"2023-11-24T06:27:43.656706+0000","last_clean":"2023-11-24T06:27:43.656706+0000","last_became_active":"2023-11-24T06:27:41.384475+0000","last_became_peered":"2023-11-24T06:27:41.384475+0000","last_unstale":"2023-11-24T06:27:43.656706+0000","last_undegraded":"2023-11-24T06:27:43.656706+0000","last_fullsized":"2023-11-24T06:27:43.656706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:22.303610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357526+0000","last_change":"2023-11-24T06:27:42.381195+0000","last_active":"2023-11-24T06:27:43.357526+0000","last_peered":"2023-11-24T06:27:43.357526+0000","last_clean":"2023-11-24T06:27:43.357526+0000","last_became_active":"2023-11-24T06:27:42.377313+0000","last_became_peered":"2023-11-24T06:27:42.377313+0000","last_unstale":"2023-11-24T06:27:43.357526+0000","last_undegraded":"2023-11-24T06:27:43.357526+0000","last_fullsized":"2023-11-24T06:27:43.357526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:32.562262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657763+0000","last_change":"2023-11-24T06:27:41.377860+0000","last_active":"2023-11-24T06:27:43.657763+0000","last_peered":"2023-11-24T06:27:43.657763+0000","last_clean":"2023-11-24T06:27:43.657763+0000","last_became_active":"2023-11-24T06:27:41.377572+0000","last_became_peered":"2023-11-24T06:27:41.377572+0000","last_unstale":"2023-11-24T06:27:43.657763+0000","last_undegraded":"2023-11-24T06:27:43.657763+0000","last_fullsized":"2023-11-24T06:27:43.657763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:52.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355182+0000","last_change":"2023-11-24T06:27:42.385894+0000","last_active":"2023-11-24T06:27:43.355182+0000","last_peered":"2023-11-24T06:27:43.355182+0000","last_clean":"2023-11-24T06:27:43.355182+0000","last_became_active":"2023-11-24T06:27:42.385586+0000","last_became_peered":"2023-11-24T06:27:42.385586+0000","last_unstale":"2023-11-24T06:27:43.355182+0000","last_undegraded":"2023-11-24T06:27:43.355182+0000","last_fullsized":"2023-11-24T06:27:43.355182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:33.522071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658185+0000","last_change":"2023-11-24T06:27:41.380746+0000","last_active":"2023-11-24T06:27:43.658185+0000","last_peered":"2023-11-24T06:27:43.658185+0000","last_clean":"2023-11-24T06:27:43.658185+0000","last_became_active":"2023-11-24T06:27:41.380535+0000","last_became_peered":"2023-11-24T06:27:41.380535+0000","last_unstale":"2023-11-24T06:27:43.658185+0000","last_undegraded":"2023-11-24T06:27:43.658185+0000","last_fullsized":"2023-11-24T06:27:43.658185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:10.313348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.355700+0000","last_change":"2023-11-24T06:27:42.399877+0000","last_active":"2023-11-24T06:27:43.355700+0000","last_peered":"2023-11-24T06:27:43.355700+0000","last_clean":"2023-11-24T06:27:43.355700+0000","last_became_active":"2023-11-24T06:27:42.399051+0000","last_became_peered":"2023-11-24T06:27:42.399051+0000","last_unstale":"2023-11-24T06:27:43.355700+0000","last_undegraded":"2023-11-24T06:27:43.355700+0000","last_fullsized":"2023-11-24T06:27:43.355700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:09.167666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659116+0000","last_change":"2023-11-24T06:27:41.386935+0000","last_active":"2023-11-24T06:27:43.659116+0000","last_peered":"2023-11-24T06:27:43.659116+0000","last_clean":"2023-11-24T06:27:43.659116+0000","last_became_active":"2023-11-24T06:27:41.386607+0000","last_became_peered":"2023-11-24T06:27:41.386607+0000","last_unstale":"2023-11-24T06:27:43.659116+0000","last_undegraded":"2023-11-24T06:27:43.659116+0000","last_fullsized":"2023-11-24T06:27:43.659116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.659499+0000","last_change":"2023-11-24T06:27:42.382231+0000","last_active":"2023-11-24T06:27:43.659499+0000","last_peered":"2023-11-24T06:27:43.659499+0000","last_clean":"2023-11-24T06:27:43.659499+0000","last_became_active":"2023-11-24T06:27:42.381452+0000","last_became_peered":"2023-11-24T06:27:42.381452+0000","last_unstale":"2023-11-24T06:27:43.659499+0000","last_undegraded":"2023-11-24T06:27:43.659499+0000","last_fullsized":"2023-11-24T06:27:43.659499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:41.297789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.358206+0000","last_change":"2023-11-24T06:27:42.378166+0000","last_active":"2023-11-24T06:27:43.358206+0000","last_peered":"2023-11-24T06:27:43.358206+0000","last_clean":"2023-11-24T06:27:43.358206+0000","last_became_active":"2023-11-24T06:27:42.377859+0000","last_became_peered":"2023-11-24T06:27:42.377859+0000","last_unstale":"2023-11-24T06:27:43.358206+0000","last_undegraded":"2023-11-24T06:27:43.358206+0000","last_fullsized":"2023-11-24T06:27:43.358206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:56.664438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.657894+0000","last_change":"2023-11-24T06:27:41.386057+0000","last_active":"2023-11-24T06:27:43.657894+0000","last_peered":"2023-11-24T06:27:43.657894+0000","last_clean":"2023-11-24T06:27:43.657894+0000","last_became_active":"2023-11-24T06:27:41.385754+0000","last_became_peered":"2023-11-24T06:27:41.385754+0000","last_unstale":"2023-11-24T06:27:43.657894+0000","last_undegraded":"2023-11-24T06:27:43.657894+0000","last_fullsized":"2023-11-24T06:27:43.657894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:59.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.357781+0000","last_change":"2023-11-24T06:27:42.382235+0000","last_active":"2023-11-24T06:27:43.357781+0000","last_peered":"2023-11-24T06:27:43.357781+0000","last_clean":"2023-11-24T06:27:43.357781+0000","last_became_active":"2023-11-24T06:27:42.380559+0000","last_became_peered":"2023-11-24T06:27:42.380559+0000","last_unstale":"2023-11-24T06:27:43.357781+0000","last_undegraded":"2023-11-24T06:27:43.357781+0000","last_fullsized":"2023-11-24T06:27:43.357781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:54.984299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.658434+0000","last_change":"2023-11-24T06:27:41.385695+0000","last_active":"2023-11-24T06:27:43.658434+0000","last_peered":"2023-11-24T06:27:43.658434+0000","last_clean":"2023-11-24T06:27:43.658434+0000","last_became_active":"2023-11-24T06:27:41.385158+0000","last_became_peered":"2023-11-24T06:27:41.385158+0000","last_unstale":"2023-11-24T06:27:43.658434+0000","last_undegraded":"2023-11-24T06:27:43.658434+0000","last_fullsized":"2023-11-24T06:27:43.658434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:48.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.660564+0000","last_change":"2023-11-24T06:27:42.396117+0000","last_active":"2023-11-24T06:27:43.660564+0000","last_peered":"2023-11-24T06:27:43.660564+0000","last_clean":"2023-11-24T06:27:43.660564+0000","last_became_active":"2023-11-24T06:27:42.395511+0000","last_became_peered":"2023-11-24T06:27:42.395511+0000","last_unstale":"2023-11-24T06:27:43.660564+0000","last_undegraded":"2023-11-24T06:27:43.660564+0000","last_fullsized":"2023-11-24T06:27:43.660564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:01.374313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356285+0000","last_change":"2023-11-24T06:27:41.373126+0000","last_active":"2023-11-24T06:27:43.356285+0000","last_peered":"2023-11-24T06:27:43.356285+0000","last_clean":"2023-11-24T06:27:43.356285+0000","last_became_active":"2023-11-24T06:27:41.372858+0000","last_became_peered":"2023-11-24T06:27:41.372858+0000","last_unstale":"2023-11-24T06:27:43.356285+0000","last_undegraded":"2023-11-24T06:27:43.356285+0000","last_fullsized":"2023-11-24T06:27:43.356285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:05.485619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.356095+0000","last_change":"2023-11-24T06:27:42.386786+0000","last_active":"2023-11-24T06:27:43.356095+0000","last_peered":"2023-11-24T06:27:43.356095+0000","last_clean":"2023-11-24T06:27:43.356095+0000","last_became_active":"2023-11-24T06:27:42.386492+0000","last_became_peered":"2023-11-24T06:27:42.386492+0000","last_unstale":"2023-11-24T06:27:43.356095+0000","last_undegraded":"2023-11-24T06:27:43.356095+0000","last_fullsized":"2023-11-24T06:27:43.356095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:41.328977+0000","last_clean_scrub_stamp":"2023-11-24T06:27:41.328977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:36:11.379388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.661047+0000","last_change":"2023-11-24T06:27:41.378175+0000","last_active":"2023-11-24T06:27:43.661047+0000","last_peered":"2023-11-24T06:27:43.661047+0000","last_clean":"2023-11-24T06:27:43.661047+0000","last_became_active":"2023-11-24T06:27:41.376774+0000","last_became_peered":"2023-11-24T06:27:41.376774+0000","last_unstale":"2023-11-24T06:27:43.661047+0000","last_undegraded":"2023-11-24T06:27:43.661047+0000","last_fullsized":"2023-11-24T06:27:43.661047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:39.886192+0000","last_clean_scrub_stamp":"2023-11-24T06:27:39.886192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:17.668607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":823296,"data_stored":669752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.323}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3849999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.542}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0550000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2029999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3119999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1779999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8460000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:53.449 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:27:53.449 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T06:27:53.450 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T06:27:53.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T06:27:53.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.707+0000 7f20ebfff640 1 Processor -- start 2023-11-24T06:27:53.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.707+0000 7f20ebfff640 1 -- start start 2023-11-24T06:27:53.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.708+0000 7f20ebfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.708+0000 7f20ebfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20ec06d050 0x7f20ec06d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20ebfff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec06d9a0 con 0x7f20ec159590 2023-11-24T06:27:53.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20ebfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec071790 con 0x7f20ec06d050 2023-11-24T06:27:53.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20ebfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec071100 con 0x7f20ec0b3c20 2023-11-24T06:27:53.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20eaffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f20ec159590 0x7f20ec0ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47742/0 (socket says 172.21.15.64:47742) 2023-11-24T06:27:53.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20eaffd640 1 -- 172.21.15.64:0/1944782696 learned_addr learned my addr 172.21.15.64:0/1944782696 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20eb7fe640 1 --2- 172.21.15.64:0/1944782696 >> v2:172.21.15.64:3300/0 conn(0x7f20ec06d050 0x7f20ec06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20ea7fc640 1 --2- 172.21.15.64:0/1944782696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20e9ffb640 1 -- 172.21.15.64:0/1944782696 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017764864 0 0) 0x7f20ec06d9a0 con 0x7f20ec159590 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20e9ffb640 1 -- 172.21.15.64:0/1944782696 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d4003520 con 0x7f20ec159590 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20eb7fe640 1 -- 172.21.15.64:0/1944782696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 msgr2=0x7f20ec070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20eb7fe640 1 --2- 172.21.15.64:0/1944782696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20eb7fe640 1 -- 172.21.15.64:0/1944782696 >> v1:172.21.15.64:6789/0 conn(0x7f20ec159590 legacy=0x7f20ec0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20eb7fe640 1 -- 172.21.15.64:0/1944782696 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec070860 con 0x7f20ec06d050 2023-11-24T06:27:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.709+0000 7f20ea7fc640 1 --2- 172.21.15.64:0/1944782696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.710+0000 7f20eb7fe640 1 --2- 172.21.15.64:0/1944782696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec06d050 0x7f20ec06d460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009d90 tx=0x7f20e402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520e6eca1adebf28 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20e9ffb640 1 -- 172.21.15.64:0/1944782696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e402cbc0 con 0x7f20ec06d050 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20e9ffb640 1 -- 172.21.15.64:0/1944782696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e402cd20 con 0x7f20ec06d050 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20e9ffb640 1 -- 172.21.15.64:0/1944782696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4034660 con 0x7f20ec06d050 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20ebfff640 1 -- 172.21.15.64:0/1944782696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec06d050 msgr2=0x7f20ec06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20ebfff640 1 --2- 172.21.15.64:0/1944782696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec06d050 0x7f20ec06d460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009d90 tx=0x7f20e402be10 comp rx=0 tx=0).stop 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20ebfff640 1 -- 172.21.15.64:0/1944782696 shutdown_connections 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20ebfff640 1 --2- 172.21.15.64:0/1944782696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec06d050 0x7f20ec06d460 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20ebfff640 1 --2- 172.21.15.64:0/1944782696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.711+0000 7f20ebfff640 1 -- 172.21.15.64:0/1944782696 >> 172.21.15.64:0/1944782696 conn(0x7f20ec06a800 msgr2=0x7f20ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.712+0000 7f20ebfff640 1 -- 172.21.15.64:0/1944782696 shutdown_connections 2023-11-24T06:27:53.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.712+0000 7f20ebfff640 1 -- 172.21.15.64:0/1944782696 wait complete. 2023-11-24T06:27:53.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.713+0000 7f20ebfff640 1 Processor -- start 2023-11-24T06:27:53.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.713+0000 7f20ebfff640 1 -- start start 2023-11-24T06:27:53.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.714+0000 7f20ebfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec06d050 0x7f20ec152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.714+0000 7f20ebfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.714+0000 7f20ebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec159590 0x7f20ec1ee780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20ebfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec152930 con 0x7f20ec159590 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20ebfff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec152c10 con 0x7f20ec06d050 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20ebfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec152ef0 con 0x7f20ec0b3c20 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec159590 0x7f20ec1ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec159590 0x7f20ec1ee780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41918/0 (socket says 172.21.15.64:41918) 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 -- 172.21.15.64:0/2789037555 learned_addr learned my addr 172.21.15.64:0/2789037555 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20ea7fc640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eaffd640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec06d050 0x7f20ec152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 -- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 msgr2=0x7f20ec150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 -- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec06d050 msgr2=0x7f20ec152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec06d050 0x7f20ec152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.715+0000 7f20eb7fe640 1 -- 172.21.15.64:0/2789037555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4009a70 con 0x7f20ec159590 2023-11-24T06:27:53.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.716+0000 7f20eb7fe640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec159590 0x7f20ec1ee780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009d90 tx=0x7f20e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.716+0000 7f20d37fe640 1 -- 172.21.15.64:0/2789037555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4040650 con 0x7f20ec159590 2023-11-24T06:27:53.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.716+0000 7f20d37fe640 1 -- 172.21.15.64:0/2789037555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e402ce90 con 0x7f20ec159590 2023-11-24T06:27:53.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.716+0000 7f20d37fe640 1 -- 172.21.15.64:0/2789037555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e40409f0 con 0x7f20ec159590 2023-11-24T06:27:53.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.716+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec14a710 con 0x7f20ec159590 2023-11-24T06:27:53.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.716+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec146ca0 con 0x7f20ec159590 2023-11-24T06:27:53.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.717+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec14ac30 con 0x7f20ec159590 2023-11-24T06:27:53.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.718+0000 7f20d37fe640 1 -- 172.21.15.64:0/2789037555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20e4040b50 con 0x7f20ec159590 2023-11-24T06:27:53.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.719+0000 7f20d37fe640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f20d806ae40 0x7f20d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.719+0000 7f20d37fe640 1 -- 172.21.15.64:0/2789037555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e40ac220 con 0x7f20ec159590 2023-11-24T06:27:53.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.719+0000 7f20eaffd640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f20d806ae40 0x7f20d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.720+0000 7f20eaffd640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f20d806ae40 0x7f20d806d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0011e0 tx=0x7f20dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.723+0000 7f20d37fe640 1 -- 172.21.15.64:0/2789037555 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20ec14ac30 con 0x7f20ec159590 2023-11-24T06:27:53.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.965+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f20ec14ac30 con 0x7f20ec159590 2023-11-24T06:27:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.965+0000 7f20d37fe640 1 -- 172.21.15.64:0/2789037555 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f20ec14ac30 con 0x7f20ec159590 2023-11-24T06:27:53.969 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:53.969 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T06:27:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f20d806ae40 msgr2=0x7f20d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f20d806ae40 0x7f20d806d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0011e0 tx=0x7f20dc009040 comp rx=0 tx=0).stop 2023-11-24T06:27:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec159590 msgr2=0x7f20ec1ee780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec159590 0x7f20ec1ee780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009d90 tx=0x7f20e4004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 shutdown_connections 2023-11-24T06:27:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f20d806ae40 0x7f20d806d300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ec159590 0x7f20ec1ee780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ec0b3c20 0x7f20ec150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 --2- 172.21.15.64:0/2789037555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20ec06d050 0x7f20ec152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.968+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 >> 172.21.15.64:0/2789037555 conn(0x7f20ec06a800 msgr2=0x7f20ec0ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.969+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 shutdown_connections 2023-11-24T06:27:53.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:53.969+0000 7f20ebfff640 1 -- 172.21.15.64:0/2789037555 wait complete. 2023-11-24T06:27:53.991 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T06:27:53.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.214+0000 7f1fd759e640 1 Processor -- start 2023-11-24T06:27:54.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.214+0000 7f1fd759e640 1 -- start start 2023-11-24T06:27:54.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.215+0000 7f1fd759e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1fd815d3c0 0x7f1fd80d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.215+0000 7f1fd759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fd80d0e90 0x7f1fd80afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd759e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80d12f0 con 0x7f1fd80b01b0 2023-11-24T06:27:54.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd80ce5d0 con 0x7f1fd80d0e90 2023-11-24T06:27:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd759e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd80b23c0 con 0x7f1fd815d3c0 2023-11-24T06:27:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd5d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fd80d0e90 0x7f1fd80afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd5d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fd80d0e90 0x7f1fd80afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41926/0 (socket says 172.21.15.64:41926) 2023-11-24T06:27:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd5d9b640 1 -- 172.21.15.64:0/199913631 learned_addr learned my addr 172.21.15.64:0/199913631 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd659c640 1 --2- 172.21.15.64:0/199913631 >> v2:172.21.15.162:3300/0 conn(0x7f1fd815d3c0 0x7f1fd80d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd5d9b640 1 -- 172.21.15.64:0/199913631 >> v2:172.21.15.162:3300/0 conn(0x7f1fd815d3c0 msgr2=0x7f1fd80d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd5d9b640 1 --2- 172.21.15.64:0/199913631 >> v2:172.21.15.162:3300/0 conn(0x7f1fd815d3c0 0x7f1fd80d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd5d9b640 1 -- 172.21.15.64:0/199913631 >> v1:172.21.15.162:6789/0 conn(0x7f1fd80b01b0 legacy=0x7f1fd80ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.216+0000 7f1fd5d9b640 1 -- 172.21.15.64:0/199913631 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80d0800 con 0x7f1fd80d0e90 2023-11-24T06:27:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.218+0000 7f1fd5d9b640 1 --2- 172.21.15.64:0/199913631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80d0e90 0x7f1fd80afc70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80b5bb0 tx=0x7f1fc002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e1332ce64a4015 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.218+0000 7f1fd559a640 1 -- 172.21.15.64:0/199913631 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00317e0 con 0x7f1fd80d0e90 2023-11-24T06:27:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.218+0000 7f1fd559a640 1 -- 172.21.15.64:0/199913631 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc0031940 con 0x7f1fd80d0e90 2023-11-24T06:27:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.218+0000 7f1fd559a640 1 -- 172.21.15.64:0/199913631 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0031cd0 con 0x7f1fd80d0e90 2023-11-24T06:27:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 -- 172.21.15.64:0/199913631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80d0e90 msgr2=0x7f1fd80afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 --2- 172.21.15.64:0/199913631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80d0e90 0x7f1fd80afc70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80b5bb0 tx=0x7f1fc002b880 comp rx=0 tx=0).stop 2023-11-24T06:27:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 -- 172.21.15.64:0/199913631 shutdown_connections 2023-11-24T06:27:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 --2- 172.21.15.64:0/199913631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80d0e90 0x7f1fd80afc70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 --2- 172.21.15.64:0/199913631 >> v2:172.21.15.162:3300/0 conn(0x7f1fd815d3c0 0x7f1fd80d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 -- 172.21.15.64:0/199913631 >> 172.21.15.64:0/199913631 conn(0x7f1fd80d30b0 msgr2=0x7f1fd80d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 -- 172.21.15.64:0/199913631 shutdown_connections 2023-11-24T06:27:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.219+0000 7f1fd759e640 1 -- 172.21.15.64:0/199913631 wait complete. 2023-11-24T06:27:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.221+0000 7f1fd759e640 1 Processor -- start 2023-11-24T06:27:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.221+0000 7f1fd759e640 1 -- start start 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80b01b0 0x7f1fd80b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd80d0e90 0x7f1fd80b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd815d3c0 0x7f1fd815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b5a60 con 0x7f1fd80b01b0 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b51e0 con 0x7f1fd80d0e90 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b54a0 con 0x7f1fd815d3c0 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80b01b0 0x7f1fd80b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80b01b0 0x7f1fd80b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41942/0 (socket says 172.21.15.64:41942) 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd659c640 1 -- 172.21.15.64:0/807751397 learned_addr learned my addr 172.21.15.64:0/807751397 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd6d9d640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd815d3c0 0x7f1fd815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd659c640 1 -- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd815d3c0 msgr2=0x7f1fd815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd659c640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd815d3c0 0x7f1fd815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd5d9b640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd80d0e90 0x7f1fd80b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.222+0000 7f1fd659c640 1 -- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd80d0e90 msgr2=0x7f1fd80b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fd659c640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd80d0e90 0x7f1fd80b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fd659c640 1 -- 172.21.15.64:0/807751397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00094d0 con 0x7f1fd80b01b0 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fd6d9d640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd815d3c0 0x7f1fd815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fd659c640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80b01b0 0x7f1fd80b27a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ccd0 tx=0x7f1fc800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fceffd640 1 -- 172.21.15.64:0/807751397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8003990 con 0x7f1fd80b01b0 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fceffd640 1 -- 172.21.15.64:0/807751397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc8003af0 con 0x7f1fd80b01b0 2023-11-24T06:27:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fceffd640 1 -- 172.21.15.64:0/807751397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8004020 con 0x7f1fd80b01b0 2023-11-24T06:27:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd80b5780 con 0x7f1fd80b01b0 2023-11-24T06:27:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.223+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd80cb5a0 con 0x7f1fd80b01b0 2023-11-24T06:27:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.224+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd80cb7c0 con 0x7f1fd80b01b0 2023-11-24T06:27:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.225+0000 7f1fceffd640 1 -- 172.21.15.64:0/807751397 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1fc8004180 con 0x7f1fd80b01b0 2023-11-24T06:27:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.227+0000 7f1fceffd640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1fb806ae40 0x7f1fb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.227+0000 7f1fceffd640 1 -- 172.21.15.64:0/807751397 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fc8089a40 con 0x7f1fd80b01b0 2023-11-24T06:27:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.227+0000 7f1fd5d9b640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1fb806ae40 0x7f1fb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.228+0000 7f1fd5d9b640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1fb806ae40 0x7f1fb806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0036000 tx=0x7f1fc003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.231+0000 7f1fceffd640 1 -- 172.21.15.64:0/807751397 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1fc808be60 con 0x7f1fd80b01b0 2023-11-24T06:27:54.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.425+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fd8067b60 con 0x7f1fd80b01b0 2023-11-24T06:27:54.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.427+0000 7f1fceffd640 1 -- 172.21.15.64:0/807751397 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f1fc801d070 con 0x7f1fd80b01b0 2023-11-24T06:27:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:54.431 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:54.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1fb806ae40 msgr2=0x7f1fb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1fb806ae40 0x7f1fb806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0036000 tx=0x7f1fc003d040 comp rx=0 tx=0).stop 2023-11-24T06:27:54.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80b01b0 msgr2=0x7f1fd80b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80b01b0 0x7f1fd80b27a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ccd0 tx=0x7f1fc800a5c0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 shutdown_connections 2023-11-24T06:27:54.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1fb806ae40 0x7f1fb806d300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd815d3c0 0x7f1fd815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd80d0e90 0x7f1fd80b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 --2- 172.21.15.64:0/807751397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd80b01b0 0x7f1fd80b27a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 >> 172.21.15.64:0/807751397 conn(0x7f1fd80d30b0 msgr2=0x7f1fd815db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 shutdown_connections 2023-11-24T06:27:54.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.430+0000 7f1fd759e640 1 -- 172.21.15.64:0/807751397 wait complete. 2023-11-24T06:27:54.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:54.455 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.685+0000 7f9fa359e640 1 Processor -- start 2023-11-24T06:27:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.685+0000 7f9fa359e640 1 -- start start 2023-11-24T06:27:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.686+0000 7f9fa359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.686+0000 7f9fa359e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9fa415b2e0 0x7f9fa40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.686+0000 7f9fa359e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa40b1db0 con 0x7f9fa40d0ec0 2023-11-24T06:27:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.686+0000 7f9fa359e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa40cfe00 con 0x7f9fa415b2e0 2023-11-24T06:27:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.686+0000 7f9fa359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40b11a0 con 0x7f9fa40d0ae0 2023-11-24T06:27:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.687+0000 7f9fa2d9d640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9fa40d0ec0 0x7f9fa40cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:53422/0 (socket says 172.21.15.64:53422) 2023-11-24T06:27:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.687+0000 7f9fa2d9d640 1 -- 172.21.15.64:0/2694343735 learned_addr learned my addr 172.21.15.64:0/2694343735 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.687+0000 7f9fa259c640 1 --2- 172.21.15.64:0/2694343735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.687+0000 7f9fa1d9b640 1 --2- 172.21.15.64:0/2694343735 >> v2:172.21.15.162:3300/0 conn(0x7f9fa415b2e0 0x7f9fa40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.688+0000 7f9fa159a640 1 -- 172.21.15.64:0/2694343735 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311008054 0 0) 0x7f9fa40b1db0 con 0x7f9fa40d0ec0 2023-11-24T06:27:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.688+0000 7f9fa159a640 1 -- 172.21.15.64:0/2694343735 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f90003520 con 0x7f9fa40d0ec0 2023-11-24T06:27:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.688+0000 7f9fa259c640 1 -- 172.21.15.64:0/2694343735 >> v2:172.21.15.162:3300/0 conn(0x7f9fa415b2e0 msgr2=0x7f9fa40cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.688+0000 7f9fa259c640 1 --2- 172.21.15.64:0/2694343735 >> v2:172.21.15.162:3300/0 conn(0x7f9fa415b2e0 0x7f9fa40cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.688+0000 7f9fa259c640 1 -- 172.21.15.64:0/2694343735 >> v1:172.21.15.162:6789/0 conn(0x7f9fa40d0ec0 legacy=0x7f9fa40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.688+0000 7f9fa259c640 1 -- 172.21.15.64:0/2694343735 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa40b1480 con 0x7f9fa40d0ae0 2023-11-24T06:27:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.689+0000 7f9fa259c640 1 --2- 172.21.15.64:0/2694343735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa40b0aa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009d90 tx=0x7f9f9402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859e72cc348de792 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.689+0000 7f9fa159a640 1 -- 172.21.15.64:0/2694343735 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9402cc60 con 0x7f9fa40d0ae0 2023-11-24T06:27:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.689+0000 7f9fa159a640 1 -- 172.21.15.64:0/2694343735 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9402cdc0 con 0x7f9fa40d0ae0 2023-11-24T06:27:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.689+0000 7f9fa159a640 1 -- 172.21.15.64:0/2694343735 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94034660 con 0x7f9fa40d0ae0 2023-11-24T06:27:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 -- 172.21.15.64:0/2694343735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 msgr2=0x7f9fa40b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 --2- 172.21.15.64:0/2694343735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa40b0aa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009d90 tx=0x7f9f9402beb0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 -- 172.21.15.64:0/2694343735 shutdown_connections 2023-11-24T06:27:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 --2- 172.21.15.64:0/2694343735 >> v2:172.21.15.162:3300/0 conn(0x7f9fa415b2e0 0x7f9fa40cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 --2- 172.21.15.64:0/2694343735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa40b0aa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 -- 172.21.15.64:0/2694343735 >> 172.21.15.64:0/2694343735 conn(0x7f9fa4099a60 msgr2=0x7f9fa4099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 -- 172.21.15.64:0/2694343735 shutdown_connections 2023-11-24T06:27:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.690+0000 7f9fa359e640 1 -- 172.21.15.64:0/2694343735 wait complete. 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.692+0000 7f9fa359e640 1 Processor -- start 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.692+0000 7f9fa359e640 1 -- start start 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.693+0000 7f9fa359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa4144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.693+0000 7f9fa359e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa40d0ec0 0x7f9fa40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.693+0000 7f9fa359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa415b2e0 0x7f9fa40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.693+0000 7f9fa359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa415e5c0 con 0x7f9fa415b2e0 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.693+0000 7f9fa359e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40ca620 con 0x7f9fa40d0ec0 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.693+0000 7f9fa359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa415e2e0 con 0x7f9fa40d0ae0 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa415b2e0 0x7f9fa40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa415b2e0 0x7f9fa40aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41944/0 (socket says 172.21.15.64:41944) 2023-11-24T06:27:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 -- 172.21.15.64:0/3601728835 learned_addr learned my addr 172.21.15.64:0/3601728835 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa259c640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa4144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa1d9b640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa40d0ec0 0x7f9fa40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 -- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 msgr2=0x7f9fa4144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa4144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 -- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa40d0ec0 msgr2=0x7f9fa40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa40d0ec0 0x7f9fa40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa2d9d640 1 -- 172.21.15.64:0/3601728835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94009a70 con 0x7f9fa415b2e0 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.694+0000 7f9fa1d9b640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa40d0ec0 0x7f9fa40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.695+0000 7f9fa2d9d640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa415b2e0 0x7f9fa40aedf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00c460 tx=0x7f9f9c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.695+0000 7f9f9affd640 1 -- 172.21.15.64:0/3601728835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c005ce0 con 0x7f9fa415b2e0 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.695+0000 7f9f9affd640 1 -- 172.21.15.64:0/3601728835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c005e40 con 0x7f9fa415b2e0 2023-11-24T06:27:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.695+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa40ca840 con 0x7f9fa415b2e0 2023-11-24T06:27:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.695+0000 7f9f9affd640 1 -- 172.21.15.64:0/3601728835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0072b0 con 0x7f9fa415b2e0 2023-11-24T06:27:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.695+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa41f2a70 con 0x7f9fa415b2e0 2023-11-24T06:27:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.696+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa4067b60 con 0x7f9fa415b2e0 2023-11-24T06:27:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.697+0000 7f9f9affd640 1 -- 172.21.15.64:0/3601728835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f9c007410 con 0x7f9fa415b2e0 2023-11-24T06:27:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.699+0000 7f9f9affd640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f8406ae40 0x7f9f8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.699+0000 7f9f9affd640 1 -- 172.21.15.64:0/3601728835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f9c088b60 con 0x7f9fa415b2e0 2023-11-24T06:27:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.699+0000 7f9fa259c640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f8406ae40 0x7f9f8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.700+0000 7f9fa259c640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f8406ae40 0x7f9f8406d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f94036000 tx=0x7f9f94030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.703+0000 7f9f9affd640 1 -- 172.21.15.64:0/3601728835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f9c053c20 con 0x7f9fa415b2e0 2023-11-24T06:27:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.901+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9fa40b2bd0 con 0x7f9fa415b2e0 2023-11-24T06:27:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.901+0000 7f9f9affd640 1 -- 172.21.15.64:0/3601728835 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f9c017020 con 0x7f9fa415b2e0 2023-11-24T06:27:54.905 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.904+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f8406ae40 msgr2=0x7f9f8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.904+0000 7f9fa359e640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f8406ae40 0x7f9f8406d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f94036000 tx=0x7f9f94030000 comp rx=0 tx=0).stop 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.904+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa415b2e0 msgr2=0x7f9fa40aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.904+0000 7f9fa359e640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa415b2e0 0x7f9fa40aedf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00c460 tx=0x7f9f9c00c920 comp rx=0 tx=0).stop 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 shutdown_connections 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f8406ae40 0x7f9f8406d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa415b2e0 0x7f9fa40aedf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa40d0ec0 0x7f9fa40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 --2- 172.21.15.64:0/3601728835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fa40d0ae0 0x7f9fa4144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 >> 172.21.15.64:0/3601728835 conn(0x7f9fa4099a60 msgr2=0x7f9fa40ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 shutdown_connections 2023-11-24T06:27:54.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:54.905+0000 7f9fa359e640 1 -- 172.21.15.64:0/3601728835 wait complete. 2023-11-24T06:27:54.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:55.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.153+0000 7fa30759e640 1 Processor -- start 2023-11-24T06:27:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.154+0000 7fa30759e640 1 -- start start 2023-11-24T06:27:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.155+0000 7fa30759e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa3080ccb50 0x7fa308070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.155+0000 7fa30759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 0x7fa308143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.155+0000 7fa30759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30815f4e0 con 0x7fa3080adf20 2023-11-24T06:27:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.155+0000 7fa30759e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30815f970 con 0x7fa3080ccb50 2023-11-24T06:27:55.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.155+0000 7fa30759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30815c3c0 con 0x7fa30815bf10 2023-11-24T06:27:55.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.156+0000 7fa30659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa3080adf20 0x7fa3080cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47756/0 (socket says 172.21.15.64:47756) 2023-11-24T06:27:55.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.156+0000 7fa30659c640 1 -- 172.21.15.64:0/4053753022 learned_addr learned my addr 172.21.15.64:0/4053753022 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:55.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.156+0000 7fa2fdd9b640 1 --2- 172.21.15.64:0/4053753022 >> v2:172.21.15.162:3300/0 conn(0x7fa3080ccb50 0x7fa308070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.156+0000 7fa306d9d640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 0x7fa308143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.156+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436447735 0 0) 0x7fa30815f4e0 con 0x7fa3080adf20 2023-11-24T06:27:55.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.156+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f4003520 con 0x7fa3080adf20 2023-11-24T06:27:55.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 611350330 0 0) 0x7fa2f4003520 con 0x7fa3080adf20 2023-11-24T06:27:55.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 msgr2=0x7fa308143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 0x7fa308143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 >> v2:172.21.15.162:3300/0 conn(0x7fa3080ccb50 msgr2=0x7fa308070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 --2- 172.21.15.64:0/4053753022 >> v2:172.21.15.162:3300/0 conn(0x7fa3080ccb50 0x7fa308070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa308071130 con 0x7fa3080adf20 2023-11-24T06:27:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fa2f80019f0 con 0x7fa3080adf20 2023-11-24T06:27:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 >> v1:172.21.15.64:6789/0 conn(0x7fa3080adf20 legacy=0x7fa3080cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f4003970 0x7fa2f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 0x7fa2f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa2f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30815f970 con 0x7fa2f4004d00 2023-11-24T06:27:55.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30815c3c0 con 0x7fa2f4003970 2023-11-24T06:27:55.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f400b980 con 0x7fa30815bf10 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa306d9d640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa2f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.157+0000 7fa2fdd9b640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 0x7fa2f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.158+0000 7fa306d9d640 1 -- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 msgr2=0x7fa2f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.158+0000 7fa306d9d640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa30815bf10 0x7fa2f40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.158+0000 7fa306d9d640 1 -- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f4003970 msgr2=0x7fa2f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.158+0000 7fa306d9d640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f4003970 0x7fa2f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.158+0000 7fa306d9d640 1 -- 172.21.15.64:0/4053753022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f4003520 con 0x7fa2f4004d00 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.159+0000 7fa306d9d640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa2f4005180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa300009ac0 tx=0x7fa2f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf9add5f024228d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.159+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30002c6b0 con 0x7fa2f4004d00 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.159+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30002c810 con 0x7fa2f4004d00 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.159+0000 7fa305d9b640 1 -- 172.21.15.64:0/4053753022 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30002cba0 con 0x7fa2f4004d00 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.159+0000 7fa30759e640 1 -- 172.21.15.64:0/4053753022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 msgr2=0x7fa2f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.159+0000 7fa30759e640 1 --2- 172.21.15.64:0/4053753022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa2f4005180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa300009ac0 tx=0x7fa2f40057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.160+0000 7fa306d9d640 1 -- 172.21.15.64:0/4053753022 reap_dead start 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.160+0000 7fa30759e640 1 -- 172.21.15.64:0/4053753022 shutdown_connections 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.160+0000 7fa30759e640 1 -- 172.21.15.64:0/4053753022 >> 172.21.15.64:0/4053753022 conn(0x7fa30806a7d0 msgr2=0x7fa30806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.160+0000 7fa30759e640 1 -- 172.21.15.64:0/4053753022 shutdown_connections 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.160+0000 7fa30759e640 1 -- 172.21.15.64:0/4053753022 wait complete. 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.162+0000 7fa30759e640 1 Processor -- start 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.162+0000 7fa30759e640 1 -- start start 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa30813b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa30813dc30 0x7fa30815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3081f1ca0 0x7fa3081f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30813bc10 con 0x7fa2f4004d00 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30813bef0 con 0x7fa30813dc30 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30813c1d0 con 0x7fa3081f1ca0 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa30813b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa30813b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41960/0 (socket says 172.21.15.64:41960) 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.163+0000 7fa30659c640 1 -- 172.21.15.64:0/520887292 learned_addr learned my addr 172.21.15.64:0/520887292 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa2fdd9b640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa30813dc30 0x7fa30815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa306d9d640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3081f1ca0 0x7fa3081f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa30659c640 1 -- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3081f1ca0 msgr2=0x7fa3081f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa30659c640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3081f1ca0 0x7fa3081f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa30659c640 1 -- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa30813dc30 msgr2=0x7fa30815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa30659c640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa30813dc30 0x7fa30815d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa30659c640 1 -- 172.21.15.64:0/520887292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30813c4b0 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa30659c640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa30813b310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800cb40 tx=0x7fa2f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa2ff7fe640 1 -- 172.21.15.64:0/520887292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800a890 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa2ff7fe640 1 -- 172.21.15.64:0/520887292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f800a9f0 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa2ff7fe640 1 -- 172.21.15.64:0/520887292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800ad80 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.164+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3081f2640 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.165+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30813b850 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.166+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa308143cf0 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.167+0000 7fa2ff7fe640 1 -- 172.21.15.64:0/520887292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2f80070d0 con 0x7fa2f4004d00 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.168+0000 7fa2ff7fe640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa2e806ae40 0x7fa2e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.168+0000 7fa2ff7fe640 1 -- 172.21.15.64:0/520887292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2f8088ec0 con 0x7fa2f4004d00 2023-11-24T06:27:55.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.168+0000 7fa2fdd9b640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa2e806ae40 0x7fa2e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.169+0000 7fa2fdd9b640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa2e806ae40 0x7fa2e806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa30806d810 tx=0x7fa2f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.172+0000 7fa2ff7fe640 1 -- 172.21.15.64:0/520887292 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2f8053ed0 con 0x7fa2f4004d00 2023-11-24T06:27:55.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.364+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa30813c640 con 0x7fa2f4004d00 2023-11-24T06:27:55.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.365+0000 7fa2ff7fe640 1 -- 172.21.15.64:0/520887292 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2f80180a0 con 0x7fa2f4004d00 2023-11-24T06:27:55.369 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:27:55.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa2e806ae40 msgr2=0x7fa2e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa2e806ae40 0x7fa2e806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa30806d810 tx=0x7fa2f0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 msgr2=0x7fa30813b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa30813b310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800cb40 tx=0x7fa2f800a060 comp rx=0 tx=0).stop 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 shutdown_connections 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa2e806ae40 0x7fa2e806d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3081f1ca0 0x7fa3081f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa30813dc30 0x7fa30815d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 --2- 172.21.15.64:0/520887292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f4004d00 0x7fa30813b310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 >> 172.21.15.64:0/520887292 conn(0x7fa30806a7d0 msgr2=0x7fa3080ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 shutdown_connections 2023-11-24T06:27:55.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.368+0000 7fa30759e640 1 -- 172.21.15.64:0/520887292 wait complete. 2023-11-24T06:27:55.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:55.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.621+0000 7fc844c46640 1 Processor -- start 2023-11-24T06:27:55.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.621+0000 7fc844c46640 1 -- start start 2023-11-24T06:27:55.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.622+0000 7fc844c46640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8400cfcf0 0x7fc840160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.622+0000 7fc844c46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc840160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.623+0000 7fc844c46640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400d4160 con 0x7fc8400d00d0 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.623+0000 7fc844c46640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8400ceff0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.623+0000 7fc844c46640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400cf410 con 0x7fc8400d3910 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.623+0000 7fc83ed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc8400d00d0 0x7fc8400b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47768/0 (socket says 172.21.15.64:47768) 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.623+0000 7fc83ed9d640 1 -- 172.21.15.64:0/1736929903 learned_addr learned my addr 172.21.15.64:0/1736929903 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.623+0000 7fc83e59c640 1 --2- 172.21.15.64:0/1736929903 >> v2:172.21.15.162:3300/0 conn(0x7fc8400cfcf0 0x7fc840160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.623+0000 7fc83dd9b640 1 --2- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc840160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.624+0000 7fc83d59a640 1 -- 172.21.15.64:0/1736929903 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956711703 0 0) 0x7fc8400d4160 con 0x7fc8400d00d0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.624+0000 7fc83d59a640 1 -- 172.21.15.64:0/1736929903 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc82c003520 con 0x7fc8400d00d0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.624+0000 7fc83e59c640 1 -- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 msgr2=0x7fc840160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.624+0000 7fc83e59c640 1 --2- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc840160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.624+0000 7fc83d59a640 1 -- 172.21.15.64:0/1736929903 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2168764806 0 0) 0x7fc82c003520 con 0x7fc8400d00d0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.624+0000 7fc83e59c640 1 -- 172.21.15.64:0/1736929903 >> v1:172.21.15.64:6789/0 conn(0x7fc8400d00d0 legacy=0x7fc8400b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.624+0000 7fc83e59c640 1 -- 172.21.15.64:0/1736929903 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400b2420 con 0x7fc8400cfcf0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.626+0000 7fc83e59c640 1 --2- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400cfcf0 0x7fc840160700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc830009dc0 tx=0x7fc83002bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18596d824ce3c9da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.626+0000 7fc83d59a640 1 -- 172.21.15.64:0/1736929903 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838001670 con 0x7fc8400cfcf0 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.626+0000 7fc83d59a640 1 -- 172.21.15.64:0/1736929903 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83002c7f0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.626+0000 7fc83d59a640 1 -- 172.21.15.64:0/1736929903 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83002cb80 con 0x7fc8400cfcf0 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 -- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400cfcf0 msgr2=0x7fc840160700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 --2- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400cfcf0 0x7fc840160700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc830009dc0 tx=0x7fc83002bcc0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 -- 172.21.15.64:0/1736929903 shutdown_connections 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 --2- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc840160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 --2- 172.21.15.64:0/1736929903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400cfcf0 0x7fc840160700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 -- 172.21.15.64:0/1736929903 >> 172.21.15.64:0/1736929903 conn(0x7fc84009c530 msgr2=0x7fc8400d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 -- 172.21.15.64:0/1736929903 shutdown_connections 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.627+0000 7fc844c46640 1 -- 172.21.15.64:0/1736929903 wait complete. 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.629+0000 7fc844c46640 1 Processor -- start 2023-11-24T06:27:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.629+0000 7fc844c46640 1 -- start start 2023-11-24T06:27:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.630+0000 7fc844c46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8400cfcf0 0x7fc84015bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.630+0000 7fc844c46640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400d00d0 0x7fc8400b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.630+0000 7fc844c46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc8400cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.630+0000 7fc844c46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84015b710 con 0x7fc8400cfcf0 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.630+0000 7fc844c46640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400cc090 con 0x7fc8400d00d0 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.630+0000 7fc844c46640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84015c180 con 0x7fc8400d3910 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.630+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8400cfcf0 0x7fc84015bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8400cfcf0 0x7fc84015bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41966/0 (socket says 172.21.15.64:41966) 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 -- 172.21.15.64:0/3071867102 learned_addr learned my addr 172.21.15.64:0/3071867102 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83ed9d640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc8400cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83dd9b640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400d00d0 0x7fc8400b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 -- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 msgr2=0x7fc8400cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc8400cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 -- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400d00d0 msgr2=0x7fc8400b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400d00d0 0x7fc8400b7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 -- 172.21.15.64:0/3071867102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc830009a70 con 0x7fc8400cfcf0 2023-11-24T06:27:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.631+0000 7fc83e59c640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8400cfcf0 0x7fc84015bba0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc830000f80 tx=0x7fc830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.632+0000 7fc836ffd640 1 -- 172.21.15.64:0/3071867102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830034c50 con 0x7fc8400cfcf0 2023-11-24T06:27:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.632+0000 7fc836ffd640 1 -- 172.21.15.64:0/3071867102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830034db0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.632+0000 7fc836ffd640 1 -- 172.21.15.64:0/3071867102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830040690 con 0x7fc8400cfcf0 2023-11-24T06:27:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.632+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84015b050 con 0x7fc8400cfcf0 2023-11-24T06:27:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.632+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8400b77d0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.633+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8400675d0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.634+0000 7fc836ffd640 1 -- 172.21.15.64:0/3071867102 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8300407f0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.635+0000 7fc836ffd640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc82006ae40 0x7fc82006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.635+0000 7fc836ffd640 1 -- 172.21.15.64:0/3071867102 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8300abf60 con 0x7fc8400cfcf0 2023-11-24T06:27:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.635+0000 7fc83dd9b640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc82006ae40 0x7fc82006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.636+0000 7fc83dd9b640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc82006ae40 0x7fc82006d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8400b90c0 tx=0x7fc828008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.640+0000 7fc836ffd640 1 -- 172.21.15.64:0/3071867102 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8300770a0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:55.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.837+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8400d3300 con 0x7fc8400cfcf0 2023-11-24T06:27:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.840+0000 7fc836ffd640 1 -- 172.21.15.64:0/3071867102 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc83007a6c0 con 0x7fc8400cfcf0 2023-11-24T06:27:55.844 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc82006ae40 msgr2=0x7fc82006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc82006ae40 0x7fc82006d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8400b90c0 tx=0x7fc828008040 comp rx=0 tx=0).stop 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8400cfcf0 msgr2=0x7fc84015bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8400cfcf0 0x7fc84015bba0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc830000f80 tx=0x7fc830004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 shutdown_connections 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc82006ae40 0x7fc82006d300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8400d3910 0x7fc8400cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8400d00d0 0x7fc8400b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 --2- 172.21.15.64:0/3071867102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8400cfcf0 0x7fc84015bba0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 >> 172.21.15.64:0/3071867102 conn(0x7fc84009c530 msgr2=0x7fc8400ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 shutdown_connections 2023-11-24T06:27:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:55.843+0000 7fc844c46640 1 -- 172.21.15.64:0/3071867102 wait complete. 2023-11-24T06:27:55.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:56.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.132+0000 7f82d597c640 1 Processor -- start 2023-11-24T06:27:56.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.132+0000 7f82d597c640 1 -- start start 2023-11-24T06:27:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.133+0000 7f82d597c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82d0071410 0x7f82d0146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82d597c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 0x7f82d0166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82d597c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d00b2e10 con 0x7f82d006c7c0 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82d597c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d006cc50 con 0x7f82d0071410 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82d597c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d006cec0 con 0x7f82d0071f00 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82cffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f82d006c7c0 0x7f82d00717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47770/0 (socket says 172.21.15.64:47770) 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82cffff640 1 -- 172.21.15.64:0/784572630 learned_addr learned my addr 172.21.15.64:0/784572630 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82d517b640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 0x7f82d0166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82d497a640 1 --2- 172.21.15.64:0/784572630 >> v2:172.21.15.162:3300/0 conn(0x7f82d0071410 0x7f82d0146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282272517 0 0) 0x7f82d00b2e10 con 0x7f82d006c7c0 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.134+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c0003520 con 0x7f82d006c7c0 2023-11-24T06:27:56.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3308981698 0 0) 0x7f82c0003520 con 0x7f82d006c7c0 2023-11-24T06:27:56.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 msgr2=0x7f82d0166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 0x7f82d0166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 >> v2:172.21.15.162:3300/0 conn(0x7f82d0071410 msgr2=0x7f82d0146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 --2- 172.21.15.64:0/784572630 >> v2:172.21.15.162:3300/0 conn(0x7f82d0071410 0x7f82d0146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d006d1a0 con 0x7f82d006c7c0 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f82bc001670 con 0x7f82d006c7c0 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 >> v1:172.21.15.64:6789/0 conn(0x7f82d006c7c0 legacy=0x7f82d00717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c0003970 0x7f82c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 0x7f82c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d006cc50 con 0x7f82c00047c0 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d006cec0 con 0x7f82c0003970 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000b980 con 0x7f82d0071f00 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.135+0000 7f82cffff640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82d517b640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 0x7f82c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82d497a640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c0003970 0x7f82c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82cffff640 1 -- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 msgr2=0x7f82c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82cffff640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 0x7f82c0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82cffff640 1 -- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c0003970 msgr2=0x7f82c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82cffff640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c0003970 0x7f82c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82cffff640 1 -- 172.21.15.64:0/784572630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0003520 con 0x7f82c00047c0 2023-11-24T06:27:56.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82d517b640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0071f00 0x7f82c0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:56.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.136+0000 7f82d497a640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c0003970 0x7f82c0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:56.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.137+0000 7f82cffff640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c190 tx=0x7f82c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db6fd7f3a9eec15 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.137+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc005ad0 con 0x7f82c00047c0 2023-11-24T06:27:56.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.137+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc005c50 con 0x7f82c00047c0 2023-11-24T06:27:56.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.137+0000 7f82cf7fe640 1 -- 172.21.15.64:0/784572630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc02c830 con 0x7f82c00047c0 2023-11-24T06:27:56.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.138+0000 7f82d597c640 1 -- 172.21.15.64:0/784572630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c00047c0 msgr2=0x7f82c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.138+0000 7f82d597c640 1 --2- 172.21.15.64:0/784572630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c190 tx=0x7f82c00057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.138+0000 7f82d517b640 1 -- 172.21.15.64:0/784572630 reap_dead start 2023-11-24T06:27:56.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.138+0000 7f82d597c640 1 -- 172.21.15.64:0/784572630 shutdown_connections 2023-11-24T06:27:56.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.138+0000 7f82d597c640 1 -- 172.21.15.64:0/784572630 >> 172.21.15.64:0/784572630 conn(0x7f82d006a800 msgr2=0x7f82d006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.138+0000 7f82d597c640 1 -- 172.21.15.64:0/784572630 shutdown_connections 2023-11-24T06:27:56.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.138+0000 7f82d597c640 1 -- 172.21.15.64:0/784572630 wait complete. 2023-11-24T06:27:56.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.140+0000 7f82d597c640 1 Processor -- start 2023-11-24T06:27:56.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.140+0000 7f82d597c640 1 -- start start 2023-11-24T06:27:56.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d597c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c00047c0 0x7f82d00c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d597c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0133530 0x7f82d00c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d597c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82d00c78d0 0x7f82d0134ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d597c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d01346b0 con 0x7f82d00c78d0 2023-11-24T06:27:56.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d597c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d0133a30 con 0x7f82c00047c0 2023-11-24T06:27:56.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d597c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d0133d10 con 0x7f82d0133530 2023-11-24T06:27:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82d00c78d0 0x7f82d0134ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82d00c78d0 0x7f82d0134ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41982/0 (socket says 172.21.15.64:41982) 2023-11-24T06:27:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 -- 172.21.15.64:0/4090804604 learned_addr learned my addr 172.21.15.64:0/4090804604 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d497a640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c00047c0 0x7f82d00c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 -- 172.21.15.64:0/4090804604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0133530 msgr2=0x7f82d00c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0133530 0x7f82d00c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 -- 172.21.15.64:0/4090804604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c00047c0 msgr2=0x7f82d00c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c00047c0 0x7f82d00c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.141+0000 7f82d517b640 1 -- 172.21.15.64:0/4090804604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0133ff0 con 0x7f82d00c78d0 2023-11-24T06:27:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.142+0000 7f82d517b640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82d00c78d0 0x7f82d0134ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f82d006e410 tx=0x7f82c800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.142+0000 7f82cd7fa640 1 -- 172.21.15.64:0/4090804604 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8017400 con 0x7f82d00c78d0 2023-11-24T06:27:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.142+0000 7f82cd7fa640 1 -- 172.21.15.64:0/4090804604 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c8011040 con 0x7f82d00c78d0 2023-11-24T06:27:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.142+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d01342d0 con 0x7f82d00c78d0 2023-11-24T06:27:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.142+0000 7f82cd7fa640 1 -- 172.21.15.64:0/4090804604 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8017780 con 0x7f82d00c78d0 2023-11-24T06:27:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.142+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d0135010 con 0x7f82d00c78d0 2023-11-24T06:27:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.143+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d01347f0 con 0x7f82d00c78d0 2023-11-24T06:27:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.144+0000 7f82cd7fa640 1 -- 172.21.15.64:0/4090804604 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82c80203b0 con 0x7f82d00c78d0 2023-11-24T06:27:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.145+0000 7f82cd7fa640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82b406ae40 0x7f82b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.145+0000 7f82cd7fa640 1 -- 172.21.15.64:0/4090804604 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82c8089740 con 0x7f82d00c78d0 2023-11-24T06:27:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.145+0000 7f82d497a640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82b406ae40 0x7f82b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.146+0000 7f82d497a640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82b406ae40 0x7f82b406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009a40 tx=0x7f82c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.149+0000 7f82cd7fa640 1 -- 172.21.15.64:0/4090804604 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82d01347f0 con 0x7f82d00c78d0 2023-11-24T06:27:56.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.382+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82d01347f0 con 0x7f82d00c78d0 2023-11-24T06:27:56.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.384+0000 7f82cd7fa640 1 -- 172.21.15.64:0/4090804604 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f82d01347f0 con 0x7f82d00c78d0 2023-11-24T06:27:56.388 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:56.388 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:27:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82b406ae40 msgr2=0x7f82b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82b406ae40 0x7f82b406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009a40 tx=0x7f82c4008040 comp rx=0 tx=0).stop 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82d00c78d0 msgr2=0x7f82d0134ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82d00c78d0 0x7f82d0134ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f82d006e410 tx=0x7f82c800f4a0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 shutdown_connections 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f82b406ae40 0x7f82b406d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82d00c78d0 0x7f82d0134ad0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82d0133530 0x7f82d00c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 --2- 172.21.15.64:0/4090804604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82c00047c0 0x7f82d00c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.386+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 >> 172.21.15.64:0/4090804604 conn(0x7f82d006a800 msgr2=0x7f82d00affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.387+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 shutdown_connections 2023-11-24T06:27:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.387+0000 7f82d597c640 1 -- 172.21.15.64:0/4090804604 wait complete. 2023-11-24T06:27:56.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.623+0000 7f16c6a39640 1 Processor -- start 2023-11-24T06:27:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.623+0000 7f16c6a39640 1 -- start start 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c6a39640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16c0162810 0x7f16c0071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c6a39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c0071780 0x7f16c006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c6a39640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c0071c10 con 0x7f16c0162430 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c6a39640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00702e0 con 0x7f16c0162810 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c6a39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00705c0 con 0x7f16c0071780 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c5a37640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f16c0162430 0x7f16c00b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47772/0 (socket says 172.21.15.64:47772) 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c5a37640 1 -- 172.21.15.64:0/1077669579 learned_addr learned my addr 172.21.15.64:0/1077669579 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.624+0000 7f16c5236640 1 --2- 172.21.15.64:0/1077669579 >> v2:172.21.15.64:3300/0 conn(0x7f16c0162810 0x7f16c0071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.625+0000 7f16c6238640 1 --2- 172.21.15.64:0/1077669579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c0071780 0x7f16c006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.625+0000 7f16c5236640 1 -- 172.21.15.64:0/1077669579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c0071780 msgr2=0x7f16c006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.625+0000 7f16c5236640 1 --2- 172.21.15.64:0/1077669579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c0071780 0x7f16c006c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.625+0000 7f16c5236640 1 -- 172.21.15.64:0/1077669579 >> v1:172.21.15.64:6789/0 conn(0x7f16c0162430 legacy=0x7f16c00b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.625+0000 7f16c4a35640 1 -- 172.21.15.64:0/1077669579 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574363929 0 0) 0x7f16c0071c10 con 0x7f16c0162430 2023-11-24T06:27:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.625+0000 7f16c5236640 1 -- 172.21.15.64:0/1077669579 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00708a0 con 0x7f16c0162810 2023-11-24T06:27:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.626+0000 7f16c5236640 1 --2- 172.21.15.64:0/1077669579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162810 0x7f16c0071240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16b00097a0 tx=0x7f16b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e452c92e5cfe542 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.626+0000 7f16c4a35640 1 -- 172.21.15.64:0/1077669579 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0031830 con 0x7f16c0162810 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.626+0000 7f16c4a35640 1 -- 172.21.15.64:0/1077669579 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b0031990 con 0x7f16c0162810 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.626+0000 7f16c4a35640 1 -- 172.21.15.64:0/1077669579 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0031d20 con 0x7f16c0162810 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 -- 172.21.15.64:0/1077669579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162810 msgr2=0x7f16c0071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 --2- 172.21.15.64:0/1077669579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162810 0x7f16c0071240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16b00097a0 tx=0x7f16b002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 -- 172.21.15.64:0/1077669579 shutdown_connections 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 --2- 172.21.15.64:0/1077669579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c0071780 0x7f16c006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 --2- 172.21.15.64:0/1077669579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162810 0x7f16c0071240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 -- 172.21.15.64:0/1077669579 >> 172.21.15.64:0/1077669579 conn(0x7f16c006a800 msgr2=0x7f16c006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 -- 172.21.15.64:0/1077669579 shutdown_connections 2023-11-24T06:27:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.627+0000 7f16c6a39640 1 -- 172.21.15.64:0/1077669579 wait complete. 2023-11-24T06:27:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.629+0000 7f16c6a39640 1 Processor -- start 2023-11-24T06:27:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.629+0000 7f16c6a39640 1 -- start start 2023-11-24T06:27:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c6a39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c0071780 0x7f16c0162e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c6a39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162430 0x7f16c01f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c6a39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c01f6d40 0x7f16c0152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c6a39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01592c0 con 0x7f16c0162430 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c6a39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0153950 con 0x7f16c0071780 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c6a39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01f71d0 con 0x7f16c01f6d40 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c5236640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162430 0x7f16c01f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c5236640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162430 0x7f16c01f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42004/0 (socket says 172.21.15.64:42004) 2023-11-24T06:27:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c5236640 1 -- 172.21.15.64:0/297430417 learned_addr learned my addr 172.21.15.64:0/297430417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c5a37640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c0071780 0x7f16c0162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.630+0000 7f16c6238640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c01f6d40 0x7f16c0152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c5236640 1 -- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c01f6d40 msgr2=0x7f16c0152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c5236640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c01f6d40 0x7f16c0152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c5236640 1 -- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c0071780 msgr2=0x7f16c0162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c5236640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c0071780 0x7f16c0162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c5236640 1 -- 172.21.15.64:0/297430417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0009400 con 0x7f16c0162430 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c5a37640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c0071780 0x7f16c0162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c5236640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162430 0x7f16c01f6800 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16b0034040 tx=0x7f16b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16b67fc640 1 -- 172.21.15.64:0/297430417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b003aa60 con 0x7f16c0162430 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16b67fc640 1 -- 172.21.15.64:0/297430417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b003abe0 con 0x7f16c0162430 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c0153670 con 0x7f16c0162430 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16b67fc640 1 -- 172.21.15.64:0/297430417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0044400 con 0x7f16c0162430 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.631+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c0158f50 con 0x7f16c0162430 2023-11-24T06:27:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.632+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0159470 con 0x7f16c0162430 2023-11-24T06:27:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.633+0000 7f16b67fc640 1 -- 172.21.15.64:0/297430417 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f16b004d440 con 0x7f16c0162430 2023-11-24T06:27:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.635+0000 7f16b67fc640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16a406ae40 0x7f16a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.635+0000 7f16b67fc640 1 -- 172.21.15.64:0/297430417 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16b00ab780 con 0x7f16c0162430 2023-11-24T06:27:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.635+0000 7f16c5a37640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16a406ae40 0x7f16a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.636+0000 7f16c5a37640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16a406ae40 0x7f16a406d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0012e0 tx=0x7f16bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.639+0000 7f16b67fc640 1 -- 172.21.15.64:0/297430417 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16c0159470 con 0x7f16c0162430 2023-11-24T06:27:56.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.841+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16c0159470 con 0x7f16c0162430 2023-11-24T06:27:56.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.846+0000 7f16b67fc640 1 -- 172.21.15.64:0/297430417 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f16c0159470 con 0x7f16c0162430 2023-11-24T06:27:56.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:56.850 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.849+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16a406ae40 msgr2=0x7f16a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.849+0000 7f16c6a39640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16a406ae40 0x7f16a406d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0012e0 tx=0x7f16bc009040 comp rx=0 tx=0).stop 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.849+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162430 msgr2=0x7f16c01f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.849+0000 7f16c6a39640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162430 0x7f16c01f6800 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16b0034040 tx=0x7f16b00046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 shutdown_connections 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16a406ae40 0x7f16a406d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16c01f6d40 0x7f16c0152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0162430 0x7f16c01f6800 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 --2- 172.21.15.64:0/297430417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c0071780 0x7f16c0162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 >> 172.21.15.64:0/297430417 conn(0x7f16c006a800 msgr2=0x7f16c00b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 shutdown_connections 2023-11-24T06:27:56.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:56.850+0000 7f16c6a39640 1 -- 172.21.15.64:0/297430417 wait complete. 2023-11-24T06:27:56.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:56.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.090+0000 7f6e80a39640 1 Processor -- start 2023-11-24T06:27:57.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.090+0000 7f6e80a39640 1 -- start start 2023-11-24T06:27:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.092+0000 7f6e80a39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 0x7f6e7c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.092+0000 7f6e80a39640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c06c730 con 0x7f6e7c143810 2023-11-24T06:27:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.092+0000 7f6e80a39640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0724d0 con 0x7f6e7c071e40 2023-11-24T06:27:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.092+0000 7f6e80a39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c143bf0 con 0x7f6e7c15de80 2023-11-24T06:27:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.092+0000 7f6e7b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6e7c143810 0x7f6e7c15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47776/0 (socket says 172.21.15.64:47776) 2023-11-24T06:27:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.092+0000 7f6e7b7fe640 1 -- 172.21.15.64:0/892691294 learned_addr learned my addr 172.21.15.64:0/892691294 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7bfff640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 0x7f6e7c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660798470 0 0) 0x7f6e7c06c730 con 0x7f6e7c143810 2023-11-24T06:27:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64003520 con 0x7f6e7c143810 2023-11-24T06:27:57.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 437616041 0 0) 0x7f6e64003520 con 0x7f6e7c143810 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 msgr2=0x7f6e7c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 0x7f6e7c0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 >> v1:172.21.15.162:6789/0 conn(0x7f6e7c071e40 legacy=0x7f6e7c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c06c9f0 con 0x7f6e7c143810 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f6e6c001670 con 0x7f6e7c143810 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 >> v1:172.21.15.64:6789/0 conn(0x7f6e7c143810 legacy=0x7f6e7c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e640039a0 0x7f6e64003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 0x7f6e640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e64005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.093+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c143bf0 con 0x7f6e64004d00 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400bb90 con 0x7f6e640039a0 2023-11-24T06:27:57.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400bd50 con 0x7f6e7c15de80 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7bfff640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e64005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7b7fe640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e640039a0 0x7f6e64003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7affd640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 0x7f6e640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7bfff640 1 -- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 msgr2=0x7f6e640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7bfff640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 0x7f6e640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7bfff640 1 -- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e640039a0 msgr2=0x7f6e64003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7bfff640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e640039a0 0x7f6e64003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7bfff640 1 -- 172.21.15.64:0/892691294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64003520 con 0x7f6e64004d00 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7b7fe640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e640039a0 0x7f6e64003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.094+0000 7f6e7affd640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c15de80 0x7f6e640047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.095+0000 7f6e7bfff640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e64005160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009fc0 tx=0x7f6e640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94d43e25a15f65c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7402cbf0 con 0x7f6e64004d00 2023-11-24T06:27:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7400f040 con 0x7f6e64004d00 2023-11-24T06:27:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e7a7fc640 1 -- 172.21.15.64:0/892691294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74034440 con 0x7f6e64004d00 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e80a39640 1 -- 172.21.15.64:0/892691294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 msgr2=0x7f6e64005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e80a39640 1 --2- 172.21.15.64:0/892691294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e64005160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009fc0 tx=0x7f6e640057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e7bfff640 1 -- 172.21.15.64:0/892691294 reap_dead start 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e80a39640 1 -- 172.21.15.64:0/892691294 shutdown_connections 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e80a39640 1 -- 172.21.15.64:0/892691294 >> 172.21.15.64:0/892691294 conn(0x7f6e7c06a7d0 msgr2=0x7f6e7c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.096+0000 7f6e80a39640 1 -- 172.21.15.64:0/892691294 shutdown_connections 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.097+0000 7f6e80a39640 1 -- 172.21.15.64:0/892691294 wait complete. 2023-11-24T06:27:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.099+0000 7f6e80a39640 1 Processor -- start 2023-11-24T06:27:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.099+0000 7f6e80a39640 1 -- start start 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e80a39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e7c1372b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e80a39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c13b930 0x7f6e7c13bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e80a39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7c0afcf0 0x7f6e7c0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e80a39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0722f0 con 0x7f6e64004d00 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e80a39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b1630 con 0x7f6e7c0afcf0 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e80a39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c13c2f0 con 0x7f6e7c13b930 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e7c1372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e7c1372b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42028/0 (socket says 172.21.15.64:42028) 2023-11-24T06:27:57.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e7b7fe640 1 -- 172.21.15.64:0/512394291 learned_addr learned my addr 172.21.15.64:0/512394291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e7b7fe640 1 -- 172.21.15.64:0/512394291 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c13b930 msgr2=0x7f6e7c13bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e7b7fe640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c13b930 0x7f6e7c13bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e7b7fe640 1 -- 172.21.15.64:0/512394291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7c0afcf0 msgr2=0x7f6e7c0b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.100+0000 7f6e7b7fe640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7c0afcf0 0x7f6e7c0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.101+0000 7f6e7b7fe640 1 -- 172.21.15.64:0/512394291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0b0a20 con 0x7f6e64004d00 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.101+0000 7f6e7b7fe640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e7c1372b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e760 tx=0x7f6e6c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.101+0000 7f6e63fff640 1 -- 172.21.15.64:0/512394291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c00ee70 con 0x7f6e64004d00 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.101+0000 7f6e63fff640 1 -- 172.21.15.64:0/512394291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c00c7d0 con 0x7f6e64004d00 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.101+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c0b0d00 con 0x7f6e64004d00 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.101+0000 7f6e63fff640 1 -- 172.21.15.64:0/512394291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c00cb40 con 0x7f6e64004d00 2023-11-24T06:27:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.101+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c0b12c0 con 0x7f6e64004d00 2023-11-24T06:27:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.102+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c0b1480 con 0x7f6e64004d00 2023-11-24T06:27:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.103+0000 7f6e63fff640 1 -- 172.21.15.64:0/512394291 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e6c00cca0 con 0x7f6e64004d00 2023-11-24T06:27:57.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.104+0000 7f6e63fff640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6e6806ae90 0x7f6e6806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.104+0000 7f6e63fff640 1 -- 172.21.15.64:0/512394291 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e6c0894a0 con 0x7f6e64004d00 2023-11-24T06:27:57.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.105+0000 7f6e7affd640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6e6806ae90 0x7f6e6806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.105+0000 7f6e7affd640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6e6806ae90 0x7f6e6806d350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c13e4a0 tx=0x7f6e70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.109+0000 7f6e63fff640 1 -- 172.21.15.64:0/512394291 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e6c054530 con 0x7f6e64004d00 2023-11-24T06:27:57.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.307+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e7c1377f0 con 0x7f6e64004d00 2023-11-24T06:27:57.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.308+0000 7f6e63fff640 1 -- 172.21.15.64:0/512394291 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e6c057b50 con 0x7f6e64004d00 2023-11-24T06:27:57.311 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:27:57.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6e6806ae90 msgr2=0x7f6e6806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6e6806ae90 0x7f6e6806d350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c13e4a0 tx=0x7f6e70009040 comp rx=0 tx=0).stop 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 msgr2=0x7f6e7c1372b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e7c1372b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e760 tx=0x7f6e6c00ec20 comp rx=0 tx=0).stop 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 shutdown_connections 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6e6806ae90 0x7f6e6806d350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7c0afcf0 0x7f6e7c0b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7c13b930 0x7f6e7c13bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 --2- 172.21.15.64:0/512394291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e64004d00 0x7f6e7c1372b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.311+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 >> 172.21.15.64:0/512394291 conn(0x7f6e7c06a7d0 msgr2=0x7f6e7c071560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.312+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 shutdown_connections 2023-11-24T06:27:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.312+0000 7f6e80a39640 1 -- 172.21.15.64:0/512394291 wait complete. 2023-11-24T06:27:57.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:57.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.548+0000 7f3261b3e640 1 Processor -- start 2023-11-24T06:27:57.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.548+0000 7f3261b3e640 1 -- start start 2023-11-24T06:27:57.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.549+0000 7f3261b3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 0x7f325c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.549+0000 7f3261b3e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325c06cea0 con 0x7f325c09a170 2023-11-24T06:27:57.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.549+0000 7f3261b3e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325c06cbc0 con 0x7f325c06c6b0 2023-11-24T06:27:57.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.549+0000 7f3261b3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c06f350 con 0x7f325c0cd240 2023-11-24T06:27:57.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.550+0000 7f325bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f325c09a170 0x7f325c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47792/0 (socket says 172.21.15.64:47792) 2023-11-24T06:27:57.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.550+0000 7f325bfff640 1 -- 172.21.15.64:0/531732034 learned_addr learned my addr 172.21.15.64:0/531732034 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:27:57.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.550+0000 7f325b7fe640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 0x7f325c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.550+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953921476 0 0) 0x7f325c06cea0 con 0x7f325c09a170 2023-11-24T06:27:57.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.550+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3248003520 con 0x7f325c09a170 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378421996 0 0) 0x7f3248003520 con 0x7f325c09a170 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 msgr2=0x7f325c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 0x7f325c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 >> v1:172.21.15.162:6789/0 conn(0x7f325c06c6b0 legacy=0x7f325c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c06f630 con 0x7f325c09a170 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f3254001670 con 0x7f325c09a170 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 >> v1:172.21.15.64:6789/0 conn(0x7f325c09a170 legacy=0x7f325c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 0x7f32480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f32480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3248004d00 0x7f3248005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c06f350 con 0x7f3248003f70 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324800bac0 con 0x7f3248004d00 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324800bc80 con 0x7f325c0cd240 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325affd640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f32480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.551+0000 7f325b7fe640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 0x7f32480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.552+0000 7f325affd640 1 -- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 msgr2=0x7f32480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.552+0000 7f325affd640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0cd240 0x7f32480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.552+0000 7f325affd640 1 -- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3248004d00 msgr2=0x7f3248005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.552+0000 7f325affd640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3248004d00 0x7f3248005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.552+0000 7f325affd640 1 -- 172.21.15.64:0/531732034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248003520 con 0x7f3248003f70 2023-11-24T06:27:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.553+0000 7f325affd640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f32480047c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3244001720 tx=0x7f32480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abca65683a81ea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.553+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440052d0 con 0x7f3248003f70 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.553+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324402e050 con 0x7f3248003f70 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.553+0000 7f325a7fc640 1 -- 172.21.15.64:0/531732034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244003830 con 0x7f3248003f70 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.554+0000 7f3261b3e640 1 -- 172.21.15.64:0/531732034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 msgr2=0x7f32480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.554+0000 7f3261b3e640 1 --2- 172.21.15.64:0/531732034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f32480047c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3244001720 tx=0x7f32480057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.554+0000 7f325bfff640 1 -- 172.21.15.64:0/531732034 reap_dead start 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.554+0000 7f3261b3e640 1 -- 172.21.15.64:0/531732034 shutdown_connections 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.554+0000 7f3261b3e640 1 -- 172.21.15.64:0/531732034 >> 172.21.15.64:0/531732034 conn(0x7f325c0b2d00 msgr2=0x7f325c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.554+0000 7f3261b3e640 1 -- 172.21.15.64:0/531732034 shutdown_connections 2023-11-24T06:27:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.554+0000 7f3261b3e640 1 -- 172.21.15.64:0/531732034 wait complete. 2023-11-24T06:27:57.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.556+0000 7f3261b3e640 1 Processor -- start 2023-11-24T06:27:57.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.556+0000 7f3261b3e640 1 -- start start 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f3261b3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f325c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f3261b3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0c4e40 0x7f325c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f3261b3e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ca280 0x7f325c0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f3261b3e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c0b4c70 con 0x7f3248003f70 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f3261b3e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c0b4060 con 0x7f325c0ca280 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f3261b3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c0b4340 con 0x7f325c0c4e40 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f325b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f325c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f325b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f325c15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42042/0 (socket says 172.21.15.64:42042) 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f325b7fe640 1 -- 172.21.15.64:0/3438400960 learned_addr learned my addr 172.21.15.64:0/3438400960 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f325affd640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0c4e40 0x7f325c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f325b7fe640 1 -- 172.21.15.64:0/3438400960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0c4e40 msgr2=0x7f325c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.557+0000 7f325b7fe640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0c4e40 0x7f325c0c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f325b7fe640 1 -- 172.21.15.64:0/3438400960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ca280 msgr2=0x7f325c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f325b7fe640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ca280 0x7f325c0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f325b7fe640 1 -- 172.21.15.64:0/3438400960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c0b4620 con 0x7f3248003f70 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f325b7fe640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f325c15d900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f324c007f30 tx=0x7f324c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f3253fff640 1 -- 172.21.15.64:0/3438400960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c00e030 con 0x7f3248003f70 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f3253fff640 1 -- 172.21.15.64:0/3438400960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c016470 con 0x7f3248003f70 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f3253fff640 1 -- 172.21.15.64:0/3438400960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c016820 con 0x7f3248003f70 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c0b4900 con 0x7f3248003f70 2023-11-24T06:27:57.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.558+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c15de40 con 0x7f3248003f70 2023-11-24T06:27:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.559+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c0b4a90 con 0x7f3248003f70 2023-11-24T06:27:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.560+0000 7f3253fff640 1 -- 172.21.15.64:0/3438400960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f324c016980 con 0x7f3248003f70 2023-11-24T06:27:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.562+0000 7f3253fff640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f323c06ae40 0x7f323c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.562+0000 7f3253fff640 1 -- 172.21.15.64:0/3438400960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f324c00f070 con 0x7f3248003f70 2023-11-24T06:27:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.562+0000 7f325affd640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f323c06ae40 0x7f323c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.563+0000 7f325affd640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f323c06ae40 0x7f323c06d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3244033000 tx=0x7f3244009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.566+0000 7f3253fff640 1 -- 172.21.15.64:0/3438400960 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f324c0556d0 con 0x7f3248003f70 2023-11-24T06:27:57.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.764+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f325c0675d0 con 0x7f3248003f70 2023-11-24T06:27:57.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.765+0000 7f3253fff640 1 -- 172.21.15.64:0/3438400960 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f324c058cf0 con 0x7f3248003f70 2023-11-24T06:27:57.767 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:27:57.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f323c06ae40 msgr2=0x7f323c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f323c06ae40 0x7f323c06d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3244033000 tx=0x7f3244009030 comp rx=0 tx=0).stop 2023-11-24T06:27:57.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 msgr2=0x7f325c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f325c15d900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f324c007f30 tx=0x7f324c011660 comp rx=0 tx=0).stop 2023-11-24T06:27:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 shutdown_connections 2023-11-24T06:27:57.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f323c06ae40 0x7f323c06d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ca280 0x7f325c0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325c0c4e40 0x7f325c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 --2- 172.21.15.64:0/3438400960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3248003f70 0x7f325c15d900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.767+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 >> 172.21.15.64:0/3438400960 conn(0x7f325c0b2d00 msgr2=0x7f325c0cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.768+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 shutdown_connections 2023-11-24T06:27:57.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:57.768+0000 7f3261b3e640 1 -- 172.21.15.64:0/3438400960 wait complete. 2023-11-24T06:27:57.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:58.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.018+0000 7fd773533640 1 Processor -- start 2023-11-24T06:27:58.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.018+0000 7fd773533640 1 -- start start 2023-11-24T06:27:58.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.019+0000 7fd773533640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd76c13df80 0x7fd76c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.019+0000 7fd773533640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd76c13e360 0x7fd76c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.019+0000 7fd773533640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c15f9c0 0x7fd76c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd773533640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c06c470 con 0x7fd76c13df80 2023-11-24T06:27:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd773533640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c06c750 con 0x7fd76c13e360 2023-11-24T06:27:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd773533640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c06ca30 con 0x7fd76c15f9c0 2023-11-24T06:27:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd76c13df80 0x7fd76c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd76c13df80 0x7fd76c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42052/0 (socket says 172.21.15.64:42052) 2023-11-24T06:27:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 -- 172.21.15.64:0/90540174 learned_addr learned my addr 172.21.15.64:0/90540174 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd771ad0640 1 --2- 172.21.15.64:0/90540174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c15f9c0 0x7fd76c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd770ace640 1 --2- 172.21.15.64:0/90540174 >> v2:172.21.15.162:3300/0 conn(0x7fd76c13e360 0x7fd76c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 -- 172.21.15.64:0/90540174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c15f9c0 msgr2=0x7fd76c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 --2- 172.21.15.64:0/90540174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c15f9c0 0x7fd76c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 -- 172.21.15.64:0/90540174 >> v2:172.21.15.162:3300/0 conn(0x7fd76c13e360 msgr2=0x7fd76c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 --2- 172.21.15.64:0/90540174 >> v2:172.21.15.162:3300/0 conn(0x7fd76c13e360 0x7fd76c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd7712cf640 1 -- 172.21.15.64:0/90540174 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c06cd10 con 0x7fd76c13df80 2023-11-24T06:27:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.020+0000 7fd771ad0640 1 --2- 172.21.15.64:0/90540174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c15f9c0 0x7fd76c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.021+0000 7fd7712cf640 1 --2- 172.21.15.64:0/90540174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c13df80 0x7fd76c15bee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd76c06ded0 tx=0x7fd76802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89169e91b3b0a2b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd763fff640 1 -- 172.21.15.64:0/90540174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76802ca70 con 0x7fd76c13df80 2023-11-24T06:27:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd763fff640 1 -- 172.21.15.64:0/90540174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76802cbd0 con 0x7fd76c13df80 2023-11-24T06:27:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd763fff640 1 -- 172.21.15.64:0/90540174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7680343f0 con 0x7fd76c13df80 2023-11-24T06:27:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd773533640 1 -- 172.21.15.64:0/90540174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c13df80 msgr2=0x7fd76c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd773533640 1 --2- 172.21.15.64:0/90540174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c13df80 0x7fd76c15bee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd76c06ded0 tx=0x7fd76802bed0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd773533640 1 -- 172.21.15.64:0/90540174 shutdown_connections 2023-11-24T06:27:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd773533640 1 --2- 172.21.15.64:0/90540174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c15f9c0 0x7fd76c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd773533640 1 --2- 172.21.15.64:0/90540174 >> v2:172.21.15.162:3300/0 conn(0x7fd76c13e360 0x7fd76c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd773533640 1 --2- 172.21.15.64:0/90540174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c13df80 0x7fd76c15bee0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.022+0000 7fd773533640 1 -- 172.21.15.64:0/90540174 >> 172.21.15.64:0/90540174 conn(0x7fd76c06a800 msgr2=0x7fd76c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.023+0000 7fd773533640 1 -- 172.21.15.64:0/90540174 shutdown_connections 2023-11-24T06:27:58.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.023+0000 7fd773533640 1 -- 172.21.15.64:0/90540174 wait complete. 2023-11-24T06:27:58.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.024+0000 7fd773533640 1 Processor -- start 2023-11-24T06:27:58.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.024+0000 7fd773533640 1 -- start start 2023-11-24T06:27:58.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.025+0000 7fd773533640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd76c13df80 0x7fd76c1f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.025+0000 7fd773533640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c13e360 0x7fd76c0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.025+0000 7fd773533640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c15f9c0 0x7fd76c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.025+0000 7fd773533640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c15e090 con 0x7fd76c15f9c0 2023-11-24T06:27:58.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.025+0000 7fd773533640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c15d510 con 0x7fd76c13df80 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.025+0000 7fd773533640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c15d7f0 con 0x7fd76c13e360 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c15f9c0 0x7fd76c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c15f9c0 0x7fd76c0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42058/0 (socket says 172.21.15.64:42058) 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 -- 172.21.15.64:0/1221569801 learned_addr learned my addr 172.21.15.64:0/1221569801 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd7712cf640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd76c13df80 0x7fd76c1f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd770ace640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c13e360 0x7fd76c0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 -- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c13e360 msgr2=0x7fd76c0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c13e360 0x7fd76c0b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 -- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd76c13df80 msgr2=0x7fd76c1f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd76c13df80 0x7fd76c1f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.026+0000 7fd771ad0640 1 -- 172.21.15.64:0/1221569801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768009b30 con 0x7fd76c15f9c0 2023-11-24T06:27:58.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.027+0000 7fd771ad0640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c15f9c0 0x7fd76c0b3230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c7f0 tx=0x7fd76400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.027+0000 7fd761ffb640 1 -- 172.21.15.64:0/1221569801 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764004220 con 0x7fd76c15f9c0 2023-11-24T06:27:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.027+0000 7fd761ffb640 1 -- 172.21.15.64:0/1221569801 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7640043a0 con 0x7fd76c15f9c0 2023-11-24T06:27:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.027+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c15dad0 con 0x7fd76c15f9c0 2023-11-24T06:27:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.027+0000 7fd761ffb640 1 -- 172.21.15.64:0/1221569801 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764007860 con 0x7fd76c15f9c0 2023-11-24T06:27:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.027+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c1f35a0 con 0x7fd76c15f9c0 2023-11-24T06:27:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.028+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76c0675d0 con 0x7fd76c15f9c0 2023-11-24T06:27:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.029+0000 7fd761ffb640 1 -- 172.21.15.64:0/1221569801 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd764007bd0 con 0x7fd76c15f9c0 2023-11-24T06:27:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.030+0000 7fd761ffb640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd75006ae40 0x7fd75006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.030+0000 7fd761ffb640 1 -- 172.21.15.64:0/1221569801 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd764089620 con 0x7fd76c15f9c0 2023-11-24T06:27:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.030+0000 7fd7712cf640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd75006ae40 0x7fd75006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.031+0000 7fd7712cf640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd75006ae40 0x7fd75006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd768035000 tx=0x7fd768030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.034+0000 7fd761ffb640 1 -- 172.21.15.64:0/1221569801 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd764054630 con 0x7fd76c15f9c0 2023-11-24T06:27:58.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.230+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd76c06df00 con 0x7fd76c15f9c0 2023-11-24T06:27:58.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.231+0000 7fd761ffb640 1 -- 172.21.15.64:0/1221569801 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd764057c50 con 0x7fd76c15f9c0 2023-11-24T06:27:58.234 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd75006ae40 msgr2=0x7fd75006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd75006ae40 0x7fd75006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd768035000 tx=0x7fd768030000 comp rx=0 tx=0).stop 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c15f9c0 msgr2=0x7fd76c0b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c15f9c0 0x7fd76c0b3230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c7f0 tx=0x7fd76400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 shutdown_connections 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd75006ae40 0x7fd75006d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd76c15f9c0 0x7fd76c0b3230 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd76c13e360 0x7fd76c0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 --2- 172.21.15.64:0/1221569801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd76c13df80 0x7fd76c1f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 >> 172.21.15.64:0/1221569801 conn(0x7fd76c06a800 msgr2=0x7fd76c0af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.234+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 shutdown_connections 2023-11-24T06:27:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.235+0000 7fd773533640 1 -- 172.21.15.64:0/1221569801 wait complete. 2023-11-24T06:27:58.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.476+0000 7f2d47fff640 1 Processor -- start 2023-11-24T06:27:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.476+0000 7f2d47fff640 1 -- start start 2023-11-24T06:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d47fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d4815afb0 0x7f2d4813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d47fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d4813fae0 0x7f2d480b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d47fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4813ff30 con 0x7f2d4813cfa0 2023-11-24T06:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d47fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d480cdc40 con 0x7f2d4813fae0 2023-11-24T06:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d47fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d480cdec0 con 0x7f2d4815afb0 2023-11-24T06:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d477fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d4813fae0 0x7f2d480b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d477fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d4813fae0 0x7f2d480b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42064/0 (socket says 172.21.15.64:42064) 2023-11-24T06:27:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.478+0000 7f2d477fe640 1 -- 172.21.15.64:0/2372625856 learned_addr learned my addr 172.21.15.64:0/2372625856 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.479+0000 7f2d467fc640 1 --2- 172.21.15.64:0/2372625856 >> v2:172.21.15.162:3300/0 conn(0x7f2d4815afb0 0x7f2d4813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.479+0000 7f2d477fe640 1 -- 172.21.15.64:0/2372625856 >> v2:172.21.15.162:3300/0 conn(0x7f2d4815afb0 msgr2=0x7f2d4813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.479+0000 7f2d477fe640 1 --2- 172.21.15.64:0/2372625856 >> v2:172.21.15.162:3300/0 conn(0x7f2d4815afb0 0x7f2d4813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.479+0000 7f2d477fe640 1 -- 172.21.15.64:0/2372625856 >> v1:172.21.15.64:6789/0 conn(0x7f2d4813cfa0 legacy=0x7f2d480cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.479+0000 7f2d477fe640 1 -- 172.21.15.64:0/2372625856 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480ce1a0 con 0x7f2d4813fae0 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.480+0000 7f2d477fe640 1 --2- 172.21.15.64:0/2372625856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4813fae0 0x7f2d480b2040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d480b3fd0 tx=0x7f2d4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00360ebbb95dc05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.480+0000 7f2d45ffb640 1 -- 172.21.15.64:0/2372625856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4002ca80 con 0x7f2d4813fae0 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.480+0000 7f2d45ffb640 1 -- 172.21.15.64:0/2372625856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4002cbe0 con 0x7f2d4813fae0 2023-11-24T06:27:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.480+0000 7f2d45ffb640 1 -- 172.21.15.64:0/2372625856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400343f0 con 0x7f2d4813fae0 2023-11-24T06:27:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 -- 172.21.15.64:0/2372625856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4813fae0 msgr2=0x7f2d480b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2372625856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4813fae0 0x7f2d480b2040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d480b3fd0 tx=0x7f2d4002bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 -- 172.21.15.64:0/2372625856 shutdown_connections 2023-11-24T06:27:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2372625856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4813fae0 0x7f2d480b2040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2372625856 >> v2:172.21.15.162:3300/0 conn(0x7f2d4815afb0 0x7f2d4813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 -- 172.21.15.64:0/2372625856 >> 172.21.15.64:0/2372625856 conn(0x7f2d4806a800 msgr2=0x7f2d4806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 -- 172.21.15.64:0/2372625856 shutdown_connections 2023-11-24T06:27:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.481+0000 7f2d47fff640 1 -- 172.21.15.64:0/2372625856 wait complete. 2023-11-24T06:27:58.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.483+0000 7f2d47fff640 1 Processor -- start 2023-11-24T06:27:58.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.483+0000 7f2d47fff640 1 -- start start 2023-11-24T06:27:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d47fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4813cfa0 0x7f2d4814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d47fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4813fae0 0x7f2d481f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4815afb0 0x7f2d480ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d47fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d48139780 con 0x7f2d4815afb0 2023-11-24T06:27:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d47fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d48159290 con 0x7f2d4813fae0 2023-11-24T06:27:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d47fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d481589d0 con 0x7f2d4813cfa0 2023-11-24T06:27:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4815afb0 0x7f2d480ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4815afb0 0x7f2d480ad880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42066/0 (socket says 172.21.15.64:42066) 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.484+0000 7f2d477fe640 1 -- 172.21.15.64:0/2114052835 learned_addr learned my addr 172.21.15.64:0/2114052835 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d477fe640 1 -- 172.21.15.64:0/2114052835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4813cfa0 msgr2=0x7f2d4814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d477fe640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4813cfa0 0x7f2d4814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d477fe640 1 -- 172.21.15.64:0/2114052835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4813fae0 msgr2=0x7f2d481f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d477fe640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4813fae0 0x7f2d481f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d477fe640 1 -- 172.21.15.64:0/2114052835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40009b30 con 0x7f2d4815afb0 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d477fe640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4815afb0 0x7f2d480ad880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d40036040 tx=0x7f2d40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d2f7fe640 1 -- 172.21.15.64:0/2114052835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40009510 con 0x7f2d4815afb0 2023-11-24T06:27:58.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d2f7fe640 1 -- 172.21.15.64:0/2114052835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4003e450 con 0x7f2d4815afb0 2023-11-24T06:27:58.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d2f7fe640 1 -- 172.21.15.64:0/2114052835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4003e800 con 0x7f2d4815afb0 2023-11-24T06:27:58.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d481394a0 con 0x7f2d4815afb0 2023-11-24T06:27:58.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.485+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d48158f20 con 0x7f2d4815afb0 2023-11-24T06:27:58.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.487+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d48159440 con 0x7f2d4815afb0 2023-11-24T06:27:58.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.487+0000 7f2d2f7fe640 1 -- 172.21.15.64:0/2114052835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d4002cd50 con 0x7f2d4815afb0 2023-11-24T06:27:58.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.488+0000 7f2d2f7fe640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3406ae40 0x7f2d3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.488+0000 7f2d2f7fe640 1 -- 172.21.15.64:0/2114052835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d400ac130 con 0x7f2d4815afb0 2023-11-24T06:27:58.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.489+0000 7f2d46ffd640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3406ae40 0x7f2d3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.489+0000 7f2d46ffd640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3406ae40 0x7f2d3406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000f60 tx=0x7f2d380079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.493+0000 7f2d2f7fe640 1 -- 172.21.15.64:0/2114052835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d48159440 con 0x7f2d4815afb0 2023-11-24T06:27:58.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.724+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d48159400 con 0x7f2d4815afb0 2023-11-24T06:27:58.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.724+0000 7f2d2f7fe640 1 -- 172.21.15.64:0/2114052835 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f2d400771d0 con 0x7f2d4815afb0 2023-11-24T06:27:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:58.727 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:58.728 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:27:58.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.727+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3406ae40 msgr2=0x7f2d3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.727+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3406ae40 0x7f2d3406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000f60 tx=0x7f2d380079b0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.727+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4815afb0 msgr2=0x7f2d480ad880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.727+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4815afb0 0x7f2d480ad880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d40036040 tx=0x7f2d40004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 shutdown_connections 2023-11-24T06:27:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3406ae40 0x7f2d3406d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d4815afb0 0x7f2d480ad880 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4813fae0 0x7f2d481f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 --2- 172.21.15.64:0/2114052835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4813cfa0 0x7f2d4814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 >> 172.21.15.64:0/2114052835 conn(0x7f2d4806a800 msgr2=0x7f2d4815a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 shutdown_connections 2023-11-24T06:27:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.728+0000 7f2d47fff640 1 -- 172.21.15.64:0/2114052835 wait complete. 2023-11-24T06:27:58.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:58.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.964+0000 7f0213590640 1 Processor -- start 2023-11-24T06:27:58.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.964+0000 7f0213590640 1 -- start start 2023-11-24T06:27:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.965+0000 7f0213590640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f020c0cfcf0 0x7f020c160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.965+0000 7f0213590640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.965+0000 7f0213590640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f020c0d00d0 0x7f020c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.965+0000 7f0213590640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0ceff0 con 0x7f020c0d00d0 2023-11-24T06:27:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.965+0000 7f0213590640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0cf410 con 0x7f020c0cfcf0 2023-11-24T06:27:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.965+0000 7f0213590640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0d4160 con 0x7f020c0d3910 2023-11-24T06:27:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f020c0d00d0 0x7f020c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f020c0d00d0 0x7f020c0b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42082/0 (socket says 172.21.15.64:42082) 2023-11-24T06:27:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 -- 172.21.15.64:0/3950591314 learned_addr learned my addr 172.21.15.64:0/3950591314 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f021132c640 1 --2- 172.21.15.64:0/3950591314 >> v2:172.21.15.162:3300/0 conn(0x7f020c0cfcf0 0x7f020c160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0210b2b640 1 --2- 172.21.15.64:0/3950591314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 -- 172.21.15.64:0/3950591314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 msgr2=0x7f020c160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 --2- 172.21.15.64:0/3950591314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 -- 172.21.15.64:0/3950591314 >> v2:172.21.15.162:3300/0 conn(0x7f020c0cfcf0 msgr2=0x7f020c160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 --2- 172.21.15.64:0/3950591314 >> v2:172.21.15.162:3300/0 conn(0x7f020c0cfcf0 0x7f020c160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0211b2d640 1 -- 172.21.15.64:0/3950591314 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0b2420 con 0x7f020c0d00d0 2023-11-24T06:27:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f0210b2b640 1 --2- 172.21.15.64:0/3950591314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.966+0000 7f021132c640 1 --2- 172.21.15.64:0/3950591314 >> v2:172.21.15.162:3300/0 conn(0x7f020c0cfcf0 0x7f020c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.967+0000 7f0211b2d640 1 --2- 172.21.15.64:0/3950591314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c0b2a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f020c0b4cc0 tx=0x7f020402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388547b58a39583d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0203fff640 1 -- 172.21.15.64:0/3950591314 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040317d0 con 0x7f020c0d00d0 2023-11-24T06:27:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0203fff640 1 -- 172.21.15.64:0/3950591314 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0204031930 con 0x7f020c0d00d0 2023-11-24T06:27:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0203fff640 1 -- 172.21.15.64:0/3950591314 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204031cc0 con 0x7f020c0d00d0 2023-11-24T06:27:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0213590640 1 -- 172.21.15.64:0/3950591314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 msgr2=0x7f020c0b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0213590640 1 --2- 172.21.15.64:0/3950591314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c0b2a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f020c0b4cc0 tx=0x7f020402b870 comp rx=0 tx=0).stop 2023-11-24T06:27:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0213590640 1 -- 172.21.15.64:0/3950591314 shutdown_connections 2023-11-24T06:27:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0213590640 1 --2- 172.21.15.64:0/3950591314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c0b2a50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0213590640 1 --2- 172.21.15.64:0/3950591314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0213590640 1 --2- 172.21.15.64:0/3950591314 >> v2:172.21.15.162:3300/0 conn(0x7f020c0cfcf0 0x7f020c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.968+0000 7f0213590640 1 -- 172.21.15.64:0/3950591314 >> 172.21.15.64:0/3950591314 conn(0x7f020c09c530 msgr2=0x7f020c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.969+0000 7f0213590640 1 -- 172.21.15.64:0/3950591314 shutdown_connections 2023-11-24T06:27:58.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.969+0000 7f0213590640 1 -- 172.21.15.64:0/3950591314 wait complete. 2023-11-24T06:27:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.970+0000 7f0213590640 1 Processor -- start 2023-11-24T06:27:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.970+0000 7f0213590640 1 -- start start 2023-11-24T06:27:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.971+0000 7f0213590640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020c0cfcf0 0x7f020c1f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.971+0000 7f0213590640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.971+0000 7f0213590640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.971+0000 7f0213590640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1596f0 con 0x7f020c0d00d0 2023-11-24T06:27:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.971+0000 7f0213590640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c159920 con 0x7f020c0cfcf0 2023-11-24T06:27:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.971+0000 7f0213590640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c159c00 con 0x7f020c0d3910 2023-11-24T06:27:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c1591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42094/0 (socket says 172.21.15.64:42094) 2023-11-24T06:27:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 -- 172.21.15.64:0/1355120388 learned_addr learned my addr 172.21.15.64:0/1355120388 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f021132c640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020c0cfcf0 0x7f020c1f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0211b2d640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 -- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 msgr2=0x7f020c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 -- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020c0cfcf0 msgr2=0x7f020c1f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020c0cfcf0 0x7f020c1f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 -- 172.21.15.64:0/1355120388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040094d0 con 0x7f020c0d00d0 2023-11-24T06:27:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0210b2b640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c1591b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c7f0 tx=0x7f01fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.972+0000 7f0201ffb640 1 -- 172.21.15.64:0/1355120388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc007a30 con 0x7f020c0d00d0 2023-11-24T06:27:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.973+0000 7f0201ffb640 1 -- 172.21.15.64:0/1355120388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc007b90 con 0x7f020c0d00d0 2023-11-24T06:27:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.973+0000 7f0201ffb640 1 -- 172.21.15.64:0/1355120388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00ce40 con 0x7f020c0d00d0 2023-11-24T06:27:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.973+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c159ee0 con 0x7f020c0d00d0 2023-11-24T06:27:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.973+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c15a1c0 con 0x7f020c0d00d0 2023-11-24T06:27:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.974+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c0675d0 con 0x7f020c0d00d0 2023-11-24T06:27:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.975+0000 7f0201ffb640 1 -- 172.21.15.64:0/1355120388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01fc0113f0 con 0x7f020c0d00d0 2023-11-24T06:27:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.975+0000 7f0201ffb640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01f006ae40 0x7f01f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.976+0000 7f0201ffb640 1 -- 172.21.15.64:0/1355120388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01fc0897c0 con 0x7f020c0d00d0 2023-11-24T06:27:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.976+0000 7f021132c640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01f006ae40 0x7f01f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.976+0000 7f021132c640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01f006ae40 0x7f01f006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0208007fc0 tx=0x7f0208007550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:58.980+0000 7f0201ffb640 1 -- 172.21.15.64:0/1355120388 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01fc054860 con 0x7f020c0d00d0 2023-11-24T06:27:59.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.174+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f020c0b1bf0 con 0x7f020c0d00d0 2023-11-24T06:27:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.178+0000 7f0201ffb640 1 -- 172.21.15.64:0/1355120388 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f01fc057e80 con 0x7f020c0d00d0 2023-11-24T06:27:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:59.181 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01f006ae40 msgr2=0x7f01f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01f006ae40 0x7f01f006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0208007fc0 tx=0x7f0208007550 comp rx=0 tx=0).stop 2023-11-24T06:27:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 msgr2=0x7f020c1591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c1591b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c7f0 tx=0x7f01fc00cbf0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 shutdown_connections 2023-11-24T06:27:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01f006ae40 0x7f01f006d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020c0d3910 0x7f020c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0d00d0 0x7f020c1591b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 --2- 172.21.15.64:0/1355120388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020c0cfcf0 0x7f020c1f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 >> 172.21.15.64:0/1355120388 conn(0x7f020c09c530 msgr2=0x7f020c06c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.180+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 shutdown_connections 2023-11-24T06:27:59.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.181+0000 7f0213590640 1 -- 172.21.15.64:0/1355120388 wait complete. 2023-11-24T06:27:59.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.423+0000 7f6bef0f8640 1 Processor -- start 2023-11-24T06:27:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.424+0000 7f6bef0f8640 1 -- start start 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.425+0000 7f6bef0f8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6be81449e0 0x7f6be80cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.425+0000 7f6bef0f8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be806ea40 0x7f6be815b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.425+0000 7f6bef0f8640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be8144dc0 con 0x7f6be815bab0 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.425+0000 7f6bef0f8640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be806f9c0 con 0x7f6be81449e0 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.425+0000 7f6bef0f8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be806ee80 con 0x7f6be806ea40 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee0f6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6be81449e0 0x7f6be80cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee0f6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6be81449e0 0x7f6be80cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42100/0 (socket says 172.21.15.64:42100) 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee0f6640 1 -- 172.21.15.64:0/177253875 learned_addr learned my addr 172.21.15.64:0/177253875 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee8f7640 1 --1- 172.21.15.64:0/177253875 >> v1:172.21.15.64:6789/0 conn(0x7f6be815bab0 0x7f6be806a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47812/0 (socket says 172.21.15.64:47812) 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bed8f5640 1 --2- 172.21.15.64:0/177253875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be806ea40 0x7f6be815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee0f6640 1 -- 172.21.15.64:0/177253875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be806ea40 msgr2=0x7f6be815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee0f6640 1 --2- 172.21.15.64:0/177253875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be806ea40 0x7f6be815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee0f6640 1 -- 172.21.15.64:0/177253875 >> v1:172.21.15.64:6789/0 conn(0x7f6be815bab0 legacy=0x7f6be806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bee0f6640 1 -- 172.21.15.64:0/177253875 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be806f120 con 0x7f6be81449e0 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.426+0000 7f6bed0f4640 1 -- 172.21.15.64:0/177253875 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890144082 0 0) 0x7f6be8144dc0 con 0x7f6be815bab0 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.427+0000 7f6bee0f6640 1 --2- 172.21.15.64:0/177253875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be81449e0 0x7f6be80cc920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009870 tx=0x7f6be402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc69c7bd17eddb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bed0f4640 1 -- 172.21.15.64:0/177253875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4031640 con 0x7f6be81449e0 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bed0f4640 1 -- 172.21.15.64:0/177253875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be40317a0 con 0x7f6be81449e0 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bed0f4640 1 -- 172.21.15.64:0/177253875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4031b30 con 0x7f6be81449e0 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 -- 172.21.15.64:0/177253875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be81449e0 msgr2=0x7f6be80cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/177253875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be81449e0 0x7f6be80cc920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009870 tx=0x7f6be402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 -- 172.21.15.64:0/177253875 shutdown_connections 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/177253875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be806ea40 0x7f6be815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/177253875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be81449e0 0x7f6be80cc920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 -- 172.21.15.64:0/177253875 >> 172.21.15.64:0/177253875 conn(0x7f6be80b1cd0 msgr2=0x7f6be80b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 -- 172.21.15.64:0/177253875 shutdown_connections 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.428+0000 7f6bef0f8640 1 -- 172.21.15.64:0/177253875 wait complete. 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.430+0000 7f6bef0f8640 1 Processor -- start 2023-11-24T06:27:59.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.430+0000 7f6bef0f8640 1 -- start start 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bef0f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be806ea40 0x7f6be81f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bef0f8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be81449e0 0x7f6be80c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bef0f8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be815bab0 0x7f6be80c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bef0f8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be80b50b0 con 0x7f6be806ea40 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bef0f8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81f6100 con 0x7f6be815bab0 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bef0f8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be80b3c60 con 0x7f6be81449e0 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bee0f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be806ea40 0x7f6be81f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.431+0000 7f6bee0f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be806ea40 0x7f6be81f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42114/0 (socket says 172.21.15.64:42114) 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee0f6640 1 -- 172.21.15.64:0/3760713382 learned_addr learned my addr 172.21.15.64:0/3760713382 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee8f7640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be815bab0 0x7f6be80c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bed8f5640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be81449e0 0x7f6be80c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee0f6640 1 -- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be81449e0 msgr2=0x7f6be80c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee0f6640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be81449e0 0x7f6be80c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee0f6640 1 -- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be815bab0 msgr2=0x7f6be80c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee0f6640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be815bab0 0x7f6be80c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee0f6640 1 -- 172.21.15.64:0/3760713382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40094d0 con 0x7f6be806ea40 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee8f7640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be815bab0 0x7f6be80c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bee0f6640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be806ea40 0x7f6be81f5bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.432+0000 7f6bdeffd640 1 -- 172.21.15.64:0/3760713382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be403ac60 con 0x7f6be806ea40 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.433+0000 7f6bdeffd640 1 -- 172.21.15.64:0/3760713382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be403ade0 con 0x7f6be806ea40 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.433+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be80b3f40 con 0x7f6be806ea40 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.433+0000 7f6bdeffd640 1 -- 172.21.15.64:0/3760713382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4044630 con 0x7f6be806ea40 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.433+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be80b4500 con 0x7f6be806ea40 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.434+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be8067450 con 0x7f6be806ea40 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.435+0000 7f6bdeffd640 1 -- 172.21.15.64:0/3760713382 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6be402f050 con 0x7f6be806ea40 2023-11-24T06:27:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.435+0000 7f6bdeffd640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6bcc06ae70 0x7f6bcc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.436+0000 7f6bdeffd640 1 -- 172.21.15.64:0/3760713382 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6be40ab4d0 con 0x7f6be806ea40 2023-11-24T06:27:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.436+0000 7f6bed8f5640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6bcc06ae70 0x7f6bcc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.437+0000 7f6bed8f5640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6bcc06ae70 0x7f6bcc06d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6be806e0e0 tx=0x7f6bd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.440+0000 7f6bdeffd640 1 -- 172.21.15.64:0/3760713382 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6be4076570 con 0x7f6be806ea40 2023-11-24T06:27:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.672+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6be806a610 con 0x7f6be806ea40 2023-11-24T06:27:59.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.673+0000 7f6bdeffd640 1 -- 172.21.15.64:0/3760713382 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f6be4031e60 con 0x7f6be806ea40 2023-11-24T06:27:59.676 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:27:59.676 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6bcc06ae70 msgr2=0x7f6bcc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6bcc06ae70 0x7f6bcc06d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6be806e0e0 tx=0x7f6bd8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be806ea40 msgr2=0x7f6be81f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be806ea40 0x7f6be81f5bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be40046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 shutdown_connections 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6bcc06ae70 0x7f6bcc06d330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be815bab0 0x7f6be80c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6be81449e0 0x7f6be80c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 --2- 172.21.15.64:0/3760713382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be806ea40 0x7f6be81f5bc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 >> 172.21.15.64:0/3760713382 conn(0x7f6be80b1cd0 msgr2=0x7f6be80a8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 shutdown_connections 2023-11-24T06:27:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:27:59.676+0000 7f6bef0f8640 1 -- 172.21.15.64:0/3760713382 wait complete. 2023-11-24T06:28:00.099 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:27:41.828633+0000', 'modified': '2023-11-24T06:27:45.282437+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4246': {'gid': 4246, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.162:6833/3583074090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6832', 'nonce': 3583074090}, {'type': 'v1', 'addr': '172.21.15.162:6833', 'nonce': 3583074090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:28:00.099 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:28:00.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-24T06:28:00.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.330+0000 7f295f59e640 1 Processor -- start 2023-11-24T06:28:00.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.330+0000 7f295f59e640 1 -- start start 2023-11-24T06:28:00.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2960162810 0x7f2960071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600702e0 con 0x7f2960162430 2023-11-24T06:28:00.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295f59e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600705c0 con 0x7f2960071780 2023-11-24T06:28:00.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295f59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29600708a0 con 0x7f2960162810 2023-11-24T06:28:00.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295e59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2960162430 0x7f29600b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47816/0 (socket says 172.21.15.64:47816) 2023-11-24T06:28:00.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295e59c640 1 -- 172.21.15.64:0/523344580 learned_addr learned my addr 172.21.15.64:0/523344580 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:00.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f2957fff640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2960162810 0x7f2960071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132846580 0 0) 0x7f29600702e0 con 0x7f2960162430 2023-11-24T06:28:00.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.332+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c003520 con 0x7f2960162430 2023-11-24T06:28:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976514383 0 0) 0x7f294c003520 con 0x7f2960162430 2023-11-24T06:28:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2960162810 msgr2=0x7f2960071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2960162810 0x7f2960071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 >> v1:172.21.15.162:6789/0 conn(0x7f2960071780 legacy=0x7f296006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2960070b80 con 0x7f2960162430 2023-11-24T06:28:00.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f2950001670 con 0x7f2960162430 2023-11-24T06:28:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 >> v1:172.21.15.64:6789/0 conn(0x7f2960162430 legacy=0x7f29600b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f294c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0047c0 0x7f294c004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c005140 0x7f294c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600708a0 con 0x7f294c0039a0 2023-11-24T06:28:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00bec0 con 0x7f294c005140 2023-11-24T06:28:00.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00c030 con 0x7f294c0047c0 2023-11-24T06:28:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.333+0000 7f295e59c640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f294c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f295ed9d640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c005140 0x7f294c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f2957fff640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0047c0 0x7f294c004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f295e59c640 1 -- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0047c0 msgr2=0x7f294c004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f295e59c640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0047c0 0x7f294c004c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f295e59c640 1 -- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c005140 msgr2=0x7f294c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f295e59c640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c005140 0x7f294c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f295e59c640 1 -- 172.21.15.64:0/523344580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c003520 con 0x7f294c0039a0 2023-11-24T06:28:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.334+0000 7f295ed9d640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c005140 0x7f294c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.335+0000 7f295e59c640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f294c003dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2950001050 tx=0x7f294c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a4c24a5f101203 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.335+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950005ad0 con 0x7f294c0039a0 2023-11-24T06:28:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.335+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2950005c30 con 0x7f294c0039a0 2023-11-24T06:28:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.335+0000 7f295dd9b640 1 -- 172.21.15.64:0/523344580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295002c7b0 con 0x7f294c0039a0 2023-11-24T06:28:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.336+0000 7f295f59e640 1 -- 172.21.15.64:0/523344580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 msgr2=0x7f294c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.336+0000 7f295f59e640 1 --2- 172.21.15.64:0/523344580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f294c003dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2950001050 tx=0x7f294c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.336+0000 7f295ed9d640 1 -- 172.21.15.64:0/523344580 reap_dead start 2023-11-24T06:28:00.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.336+0000 7f295f59e640 1 -- 172.21.15.64:0/523344580 shutdown_connections 2023-11-24T06:28:00.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.336+0000 7f295f59e640 1 -- 172.21.15.64:0/523344580 >> 172.21.15.64:0/523344580 conn(0x7f296006a800 msgr2=0x7f296006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.336+0000 7f295f59e640 1 -- 172.21.15.64:0/523344580 shutdown_connections 2023-11-24T06:28:00.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.336+0000 7f295f59e640 1 -- 172.21.15.64:0/523344580 wait complete. 2023-11-24T06:28:00.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.338+0000 7f295f59e640 1 Processor -- start 2023-11-24T06:28:00.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.338+0000 7f295f59e640 1 -- start start 2023-11-24T06:28:00.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f2960153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f296015a810 0x7f2960158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295f59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29600cc3b0 0x7f29600cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600ccd70 con 0x7f294c0039a0 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295f59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296015a050 con 0x7f29600cc3b0 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295f59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2960154020 con 0x7f296015a810 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f2960153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f2960153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42124/0 (socket says 172.21.15.64:42124) 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295e59c640 1 -- 172.21.15.64:0/3031065439 learned_addr learned my addr 172.21.15.64:0/3031065439 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295e59c640 1 -- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f296015a810 msgr2=0x7f2960158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295e59c640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f296015a810 0x7f2960158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295ed9d640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29600cc3b0 0x7f29600cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.339+0000 7f295e59c640 1 -- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29600cc3b0 msgr2=0x7f29600cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f295e59c640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29600cc3b0 0x7f29600cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f295e59c640 1 -- 172.21.15.64:0/3031065439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2960159440 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f295ed9d640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29600cc3b0 0x7f29600cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f295e59c640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f2960153ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2950000fb0 tx=0x7f2950000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f2956ffd640 1 -- 172.21.15.64:0/3031065439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950005a40 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f2956ffd640 1 -- 172.21.15.64:0/3031065439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2950007070 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f2956ffd640 1 -- 172.21.15.64:0/3031065439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950007400 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2960159720 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.340+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2960159ce0 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.343+0000 7f2956ffd640 1 -- 172.21.15.64:0/3031065439 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2950032070 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.344+0000 7f2956ffd640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f294006ae70 0x7f294006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.344+0000 7f2956ffd640 1 -- 172.21.15.64:0/3031065439 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f295003c040 con 0x7f294c0039a0 2023-11-24T06:28:00.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.344+0000 7f2957fff640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f294006ae70 0x7f294006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.344+0000 7f2967bde740 1 Processor -- start 2023-11-24T06:28:00.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.345+0000 7f2957fff640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f294006ae70 0x7f294006d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2960153440 tx=0x7f2948008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.345+0000 7f2967bde740 1 -- start start 2023-11-24T06:28:00.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b3f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2967bde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b39d70 0x55cc42b3e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b82730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2967bde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42a9d880 con 0x55cc42b39d70 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42aacae0 con 0x55cc42b3e7b0 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc42aaa1d0 con 0x55cc42b3f070 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b39d70 0x55cc42b3e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b39d70 0x55cc42b3e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42140/0 (socket says 172.21.15.64:42140) 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 -- 172.21.15.64:0/1828202513 learned_addr learned my addr 172.21.15.64:0/1828202513 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f295e59c640 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b3f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 -- 172.21.15.64:0/1828202513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 msgr2=0x55cc42b3f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b3f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 -- 172.21.15.64:0/1828202513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 msgr2=0x55cc42b82730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b82730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.346+0000 7f2957fff640 1 -- 172.21.15.64:0/1828202513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc42a6d860 con 0x55cc42b39d70 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.347+0000 7f2957fff640 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b39d70 0x55cc42b3e270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2948010870 tx=0x7f2948013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b531d366e166b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.347+0000 7f293ffff640 1 -- 172.21.15.64:0/1828202513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948015bf0 con 0x55cc42b39d70 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.347+0000 7f293ffff640 1 -- 172.21.15.64:0/1828202513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294801a070 con 0x55cc42b39d70 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.347+0000 7f293ffff640 1 -- 172.21.15.64:0/1828202513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948021440 con 0x55cc42b39d70 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.347+0000 7f2967bde740 1 -- 172.21.15.64:0/1828202513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b39d70 msgr2=0x55cc42b3e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.347+0000 7f2967bde740 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b39d70 0x55cc42b3e270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2948010870 tx=0x7f2948013000 comp rx=0 tx=0).stop 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 -- 172.21.15.64:0/1828202513 shutdown_connections 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b82730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b39d70 0x55cc42b3e270 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 --2- 172.21.15.64:0/1828202513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b3f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 -- 172.21.15.64:0/1828202513 >> 172.21.15.64:0/1828202513 conn(0x55cc42b00c70 msgr2=0x55cc42b84b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 -- 172.21.15.64:0/1828202513 shutdown_connections 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 -- 172.21.15.64:0/1828202513 wait complete. 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.348+0000 7f2967bde740 1 -- start start 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.349+0000 7f2967bde740 1 Processor -- start 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b8cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2967bde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b8a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2967bde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42aa5ac0 con 0x55cc42b3e7b0 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42a422f0 con 0x55cc42b39d70 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc42a5fc70 con 0x55cc42b3f070 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2967bde740 10 client.0 _ll_register_callbacks cb 0x55cc42a643b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b8a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b8a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42152/0 (socket says 172.21.15.64:42152) 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 -- 172.21.15.64:0/1157610926 learned_addr learned my addr 172.21.15.64:0/1157610926 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f295e59c640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b8cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f295ed9d640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 -- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 msgr2=0x55cc42b7d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b7d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 -- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 msgr2=0x55cc42b8cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b8cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f2957fff640 1 -- 172.21.15.64:0/1157610926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294800e050 con 0x55cc42b3e7b0 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.350+0000 7f295e59c640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b8cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f2957fff640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b8a650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2948023040 tx=0x7f2948009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8921bdede7f40901 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f293d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f293d7fa640 1 -- 172.21.15.64:0/1157610926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948021c00 con 0x55cc42b3e7b0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f293d7fa640 1 -- 172.21.15.64:0/1157610926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2948021d60 con 0x55cc42b3e7b0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc42a9eaa0 con 0x55cc42b3e7b0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f293d7fa640 1 -- 172.21.15.64:0/1157610926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2948004020 con 0x55cc42b3e7b0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f2967bde740 10 client.4632 fetch_fsmap learned FSMap version 8 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.351+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55cc42aa4140 con 0x55cc42b3e7b0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f293d7fa640 1 -- 172.21.15.64:0/1157610926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc42a9eaa0 con 0x55cc42b3e7b0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f293d7fa640 1 -- 172.21.15.64:0/1157610926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2948004180 con 0x55cc42b3e7b0 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f2967bde740 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f2967bde740 10 client.4632 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f2967bde740 20 client.4632 populate_metadata read hostname 'smithi064' 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f2967bde740 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b56070 0x55cc42b7bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f2967bde740 4 client.4632 mds_command: new command op to 4246 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:28:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 --> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc42a9e7c0 con 0x55cc42b56070 2023-11-24T06:28:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.352+0000 7f295e59c640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b56070 0x55cc42b7bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.353+0000 7f295e59c640 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b56070 0x55cc42b7bc10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c0dc0508598f10 server_cookie=e5091d35de9b1f9 in_seq=0 out_seq=0 2023-11-24T06:28:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.353+0000 7f293d7fa640 10 client.4632 ms_handle_connect on v2:172.21.15.162:6832/3583074090 2023-11-24T06:28:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.355+0000 7f293d7fa640 1 -- 172.21.15.64:0/1157610926 <== mds.0 v2:172.21.15.162:6832/3583074090 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc42a9e7c0 con 0x55cc42b56070 2023-11-24T06:28:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.355+0000 7f293d7fa640 10 client.4632 handle_command_reply: tid=0 2023-11-24T06:28:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.355+0000 7f2967bde740 1 client.4632 shutdown 2023-11-24T06:28:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 msgr2=0x55cc42b8a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b8a650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2948023040 tx=0x7f2948009190 comp rx=0 tx=0).stop 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 shutdown_connections 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b56070 0x55cc42b7bc10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b7d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b8a650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 --2- 172.21.15.64:0/1157610926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b8cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f293d7fa640 0 client.4632 ms_handle_reset on v2:172.21.15.162:6832/3583074090 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 >> 172.21.15.64:0/1157610926 conn(0x55cc42b00c70 msgr2=0x55cc42b38dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 shutdown_connections 2023-11-24T06:28:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 1 -- 172.21.15.64:0/1157610926 wait complete. 2023-11-24T06:28:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.356+0000 7f2967bde740 20 client.4632 trim_cache size 0 max 16384 2023-11-24T06:28:00.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.373+0000 7f2967bde740 1 Processor -- start 2023-11-24T06:28:00.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.373+0000 7f2967bde740 1 -- start start 2023-11-24T06:28:00.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b3df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2967bde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b84b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2967bde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42808f20 con 0x55cc42b3e7b0 2023-11-24T06:28:00.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42b554e0 con 0x55cc42b39d70 2023-11-24T06:28:00.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc42b55c30 con 0x55cc42b3f070 2023-11-24T06:28:00.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b84b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b84b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42158/0 (socket says 172.21.15.64:42158) 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.374+0000 7f2957fff640 1 -- 172.21.15.64:0/3017348161 learned_addr learned my addr 172.21.15.64:0/3017348161 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f295e59c640 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b3df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2957fff640 1 -- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 msgr2=0x55cc42b850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f295ed9d640 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2957fff640 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b850a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2957fff640 1 -- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 msgr2=0x55cc42b3df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2957fff640 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b3df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2957fff640 1 -- 172.21.15.64:0/3017348161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc42a41b80 con 0x55cc42b3e7b0 2023-11-24T06:28:00.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f295e59c640 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b3df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2957fff640 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b84b60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2948015970 tx=0x7f294802d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cbc530bacf6f47 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2954ff9640 1 -- 172.21.15.64:0/3017348161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948021c00 con 0x55cc42b3e7b0 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2954ff9640 1 -- 172.21.15.64:0/3017348161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948021d60 con 0x55cc42b3e7b0 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.375+0000 7f2954ff9640 1 -- 172.21.15.64:0/3017348161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948016620 con 0x55cc42b3e7b0 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 -- 172.21.15.64:0/3017348161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 msgr2=0x55cc42b84b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b84b60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2948015970 tx=0x7f294802d040 comp rx=0 tx=0).stop 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 -- 172.21.15.64:0/3017348161 shutdown_connections 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b3f070 0x55cc42b850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3e7b0 0x55cc42b84b60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 --2- 172.21.15.64:0/3017348161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b39d70 0x55cc42b3df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 -- 172.21.15.64:0/3017348161 >> 172.21.15.64:0/3017348161 conn(0x55cc42b00c70 msgr2=0x55cc42b6d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 -- 172.21.15.64:0/3017348161 shutdown_connections 2023-11-24T06:28:00.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.376+0000 7f2967bde740 1 -- 172.21.15.64:0/3017348161 wait complete. 2023-11-24T06:28:00.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.377+0000 7f2967bde740 1 -- start start 2023-11-24T06:28:00.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.377+0000 7f2967bde740 1 Processor -- start 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b39d70 0x55cc42b88480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f2967bde740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f2967bde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3f070 0x55cc42b9da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f2967bde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42b39be0 con 0x55cc42b3f070 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc42b64f40 con 0x55cc42b3e7b0 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f2967bde740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc42bee920 con 0x55cc42b39d70 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f2967bde740 10 client.0 _ll_register_callbacks cb 0x7f2948009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f295ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3f070 0x55cc42b9da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f295ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3f070 0x55cc42b9da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42162/0 (socket says 172.21.15.64:42162) 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.378+0000 7f295ed9d640 1 -- 172.21.15.64:0/118435102 learned_addr learned my addr 172.21.15.64:0/118435102 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f2957fff640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f295e59c640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b39d70 0x55cc42b88480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f295ed9d640 1 -- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b39d70 msgr2=0x55cc42b88480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f295ed9d640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b39d70 0x55cc42b88480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f295ed9d640 1 -- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 msgr2=0x55cc42b889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f295ed9d640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b889c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f295ed9d640 1 -- 172.21.15.64:0/118435102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc42aa4140 con 0x55cc42b3f070 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f2957fff640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f295ed9d640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3f070 0x55cc42b9da20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2958000f80 tx=0x7f29580021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62244e8cf2e406e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f293d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T06:28:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f293d7fa640 1 -- 172.21.15.64:0/118435102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958003b80 con 0x55cc42b3f070 2023-11-24T06:28:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f293d7fa640 1 -- 172.21.15.64:0/118435102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f295800c040 con 0x55cc42b3f070 2023-11-24T06:28:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc42b84220 con 0x55cc42b3f070 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.379+0000 7f293d7fa640 1 -- 172.21.15.64:0/118435102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2958003ce0 con 0x55cc42b3f070 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 10 client.4638 fetch_fsmap learned FSMap version 8 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55cc42a67170 con 0x55cc42b3f070 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f293d7fa640 1 -- 172.21.15.64:0/118435102 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc42b84220 con 0x55cc42b3f070 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f293d7fa640 1 -- 172.21.15.64:0/118435102 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29580050d0 con 0x55cc42b3f070 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 20 client.4638 populate_metadata read hostname 'smithi064' 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b9df60 0x55cc42bf1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 4 client.4638 mds_command: new command op to 4246 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.380+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 --> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f294801a070 con 0x55cc42b9df60 2023-11-24T06:28:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.381+0000 7f295e59c640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b9df60 0x55cc42bf1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f295e59c640 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b9df60 0x55cc42bf1f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0336f1cc0bcb750 server_cookie=8da304935c7929f7 in_seq=0 out_seq=0 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f293d7fa640 10 client.4638 ms_handle_connect on v2:172.21.15.162:6832/3583074090 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f293d7fa640 1 -- 172.21.15.64:0/118435102 <== mds.0 v2:172.21.15.162:6832/3583074090 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7f294801a070 con 0x55cc42b9df60 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f293d7fa640 10 client.4638 handle_command_reply: tid=0 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 client.4638 shutdown 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3f070 msgr2=0x55cc42b9da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3f070 0x55cc42b9da20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2958000f80 tx=0x7f29580021f0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 shutdown_connections 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:6832/3583074090,v1:172.21.15.162:6833/3583074090] conn(0x55cc42b9df60 0x55cc42bf1f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc42b3f070 0x55cc42b9da20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cc42b3e7b0 0x55cc42b889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 --2- 172.21.15.64:0/118435102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55cc42b39d70 0x55cc42b88480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 >> 172.21.15.64:0/118435102 conn(0x55cc42b00c70 msgr2=0x55cc42b7d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.382+0000 7f293d7fa640 0 client.4638 ms_handle_reset on v2:172.21.15.162:6832/3583074090 2023-11-24T06:28:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 shutdown_connections 2023-11-24T06:28:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f2967bde740 1 -- 172.21.15.64:0/118435102 wait complete. 2023-11-24T06:28:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f2967bde740 20 client.4638 trim_cache size 0 max 16384 2023-11-24T06:28:00.393 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "fb91be48-aa54-48b4-b765-8060df323172", 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4246, 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 17.625886675, 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 8, 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 22, 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 22, 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 25.930605680999999 2023-11-24T06:28:00.394 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:28:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f294006ae70 msgr2=0x7f294006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f295f59e640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f294006ae70 0x7f294006d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2960153440 tx=0x7f2948008040 comp rx=0 tx=0).stop 2023-11-24T06:28:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 msgr2=0x7f2960153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f295f59e640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f2960153ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2950000fb0 tx=0x7f2950000fe0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 shutdown_connections 2023-11-24T06:28:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f295f59e640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f294006ae70 0x7f294006d330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.383+0000 7f295f59e640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29600cc3b0 0x7f29600cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.384+0000 7f295f59e640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f296015a810 0x7f2960158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.384+0000 7f295f59e640 1 --2- 172.21.15.64:0/3031065439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c0039a0 0x7f2960153ae0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.384+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 >> 172.21.15.64:0/3031065439 conn(0x7f296006a800 msgr2=0x7f296006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.384+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 shutdown_connections 2023-11-24T06:28:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:00.384+0000 7f295f59e640 1 -- 172.21.15.64:0/3031065439 wait complete. 2023-11-24T06:28:00.403 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-24T06:28:00.428 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-24T06:28:00.429 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T06:28:00.429 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-24T06:28:00.429 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:28:00.429 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:00.429 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-24T06:28:00.429 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:28:00.429 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:00.429 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-24T06:28:00.429 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:28:00.430 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:00.430 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-24T06:28:00.430 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:28:00.430 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:00.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:00.430 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T06:28:00.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:00.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-11-24T06:28:00.599 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2023-11-24T06:28:00.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:00.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:00.603 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-11-24T06:28:00.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:00.666 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2023-11-24T06:28:00.766 INFO:teuthology.orchestra.run.smithi162.stderr:Cannot find device "ceph-brx" 2023-11-24T06:28:00.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:00.770 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-24T06:28:00.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:28:01.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.004+0000 7f9da6def640 1 Processor -- start 2023-11-24T06:28:01.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.004+0000 7f9da6def640 1 -- start start 2023-11-24T06:28:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.006+0000 7f9da6def640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 0x7f9da00cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.006+0000 7f9da6def640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da0070e50 con 0x7f9da006d160 2023-11-24T06:28:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.006+0000 7f9da6def640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da00ccea0 con 0x7f9da0143c40 2023-11-24T06:28:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.006+0000 7f9da6def640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00723c0 con 0x7f9da00cc500 2023-11-24T06:28:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.006+0000 7f9da65ee640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9da006d160 0x7f9da006d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47826/0 (socket says 172.21.15.64:47826) 2023-11-24T06:28:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.006+0000 7f9da65ee640 1 -- 172.21.15.64:0/3555860104 learned_addr learned my addr 172.21.15.64:0/3555860104 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da55ec640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 0x7f9da00cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106645357 0 0) 0x7f9da0070e50 con 0x7f9da006d160 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d8c003520 con 0x7f9da006d160 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572857712 0 0) 0x7f9d8c003520 con 0x7f9da006d160 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 msgr2=0x7f9da00cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 0x7f9da00cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 >> v1:172.21.15.162:6789/0 conn(0x7f9da0143c40 legacy=0x7f9da0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da0144020 con 0x7f9da006d160 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f9d980019f0 con 0x7f9da006d160 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 >> v1:172.21.15.64:6789/0 conn(0x7f9da006d160 legacy=0x7f9da006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c0039a0 0x7f9d8c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.007+0000 7f9da4deb640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 0x7f9d8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da4deb640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c004d00 0x7f9d8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00723c0 con 0x7f9d8c0039a0 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c00ba60 con 0x7f9d8c004d00 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c00bc20 con 0x7f9da00cc500 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da5ded640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c0039a0 0x7f9d8c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da55ec640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 0x7f9d8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da65ee640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c004d00 0x7f9d8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da5ded640 1 -- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 msgr2=0x7f9d8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da5ded640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da00cc500 0x7f9d8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da5ded640 1 -- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c004d00 msgr2=0x7f9d8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da5ded640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c004d00 0x7f9d8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.008+0000 7f9da5ded640 1 -- 172.21.15.64:0/3555860104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c003520 con 0x7f9d8c0039a0 2023-11-24T06:28:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.010+0000 7f9da5ded640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c0039a0 0x7f9d8c003d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c001860 tx=0x7f9d8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645888ae4f89f9e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.010+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c033630 con 0x7f9d8c0039a0 2023-11-24T06:28:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.010+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c0337b0 con 0x7f9d8c0039a0 2023-11-24T06:28:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.010+0000 7f9da4deb640 1 -- 172.21.15.64:0/3555860104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c033b60 con 0x7f9d8c0039a0 2023-11-24T06:28:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.011+0000 7f9da6def640 1 -- 172.21.15.64:0/3555860104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c0039a0 msgr2=0x7f9d8c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.011+0000 7f9da6def640 1 --2- 172.21.15.64:0/3555860104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c0039a0 0x7f9d8c003d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c001860 tx=0x7f9d8c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.011+0000 7f9da65ee640 1 -- 172.21.15.64:0/3555860104 reap_dead start 2023-11-24T06:28:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.011+0000 7f9da6def640 1 -- 172.21.15.64:0/3555860104 shutdown_connections 2023-11-24T06:28:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.011+0000 7f9da6def640 1 -- 172.21.15.64:0/3555860104 >> 172.21.15.64:0/3555860104 conn(0x7f9da006a800 msgr2=0x7f9da006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.011+0000 7f9da6def640 1 -- 172.21.15.64:0/3555860104 shutdown_connections 2023-11-24T06:28:01.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.011+0000 7f9da6def640 1 -- 172.21.15.64:0/3555860104 wait complete. 2023-11-24T06:28:01.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.013+0000 7f9da6def640 1 Processor -- start 2023-11-24T06:28:01.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.013+0000 7f9da6def640 1 -- start start 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.014+0000 7f9da6def640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0039a0 0x7f9da0142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.014+0000 7f9da6def640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da013fdf0 0x7f9da00acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.014+0000 7f9da6def640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da01f3630 0x7f9da0138000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.014+0000 7f9da6def640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da01f2e50 con 0x7f9da013fdf0 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.014+0000 7f9da6def640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da0142d20 con 0x7f9d8c0039a0 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.014+0000 7f9da6def640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da0143000 con 0x7f9da01f3630 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da013fdf0 0x7f9da00acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da013fdf0 0x7f9da00acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42178/0 (socket says 172.21.15.64:42178) 2023-11-24T06:28:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 -- 172.21.15.64:0/588726437 learned_addr learned my addr 172.21.15.64:0/588726437 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da5ded640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0039a0 0x7f9da0142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da65ee640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da01f3630 0x7f9da0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 -- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da01f3630 msgr2=0x7f9da0138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da01f3630 0x7f9da0138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 -- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0039a0 msgr2=0x7f9da0142710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0039a0 0x7f9da0142710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da55ec640 1 -- 172.21.15.64:0/588726437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da01432e0 con 0x7f9da013fdf0 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.015+0000 7f9da5ded640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0039a0 0x7f9da0142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.016+0000 7f9da55ec640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da013fdf0 0x7f9da00acda0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d90011600 tx=0x7f9d90011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.016+0000 7f9d967fc640 1 -- 172.21.15.64:0/588726437 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000f920 con 0x7f9da013fdf0 2023-11-24T06:28:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.016+0000 7f9d967fc640 1 -- 172.21.15.64:0/588726437 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000fa80 con 0x7f9da013fdf0 2023-11-24T06:28:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.016+0000 7f9d967fc640 1 -- 172.21.15.64:0/588726437 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000fdf0 con 0x7f9da013fdf0 2023-11-24T06:28:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.016+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da01f3110 con 0x7f9da013fdf0 2023-11-24T06:28:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.016+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da00ad2e0 con 0x7f9da013fdf0 2023-11-24T06:28:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.017+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da0067450 con 0x7f9da013fdf0 2023-11-24T06:28:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.018+0000 7f9d967fc640 1 -- 172.21.15.64:0/588726437 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d90015460 con 0x7f9da013fdf0 2023-11-24T06:28:01.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.019+0000 7f9d967fc640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d8406ae40 0x7f9d8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.019+0000 7f9d967fc640 1 -- 172.21.15.64:0/588726437 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d90089a20 con 0x7f9da013fdf0 2023-11-24T06:28:01.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.019+0000 7f9da5ded640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d8406ae40 0x7f9d8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.020+0000 7f9da5ded640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d8406ae40 0x7f9d8406d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c036000 tx=0x7f9d9c031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.023+0000 7f9d967fc640 1 -- 172.21.15.64:0/588726437 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d90054ac0 con 0x7f9da013fdf0 2023-11-24T06:28:01.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.257+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9da0144a80 con 0x7f9da013fdf0 2023-11-24T06:28:01.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.258+0000 7f9d967fc640 1 -- 172.21.15.64:0/588726437 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9d90022070 con 0x7f9da013fdf0 2023-11-24T06:28:01.261 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d8406ae40 msgr2=0x7f9d8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d8406ae40 0x7f9d8406d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c036000 tx=0x7f9d9c031000 comp rx=0 tx=0).stop 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da013fdf0 msgr2=0x7f9da00acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da013fdf0 0x7f9da00acda0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d90011600 tx=0x7f9d90011ac0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 shutdown_connections 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d8406ae40 0x7f9d8406d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9da01f3630 0x7f9da0138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da013fdf0 0x7f9da00acda0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 --2- 172.21.15.64:0/588726437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d8c0039a0 0x7f9da0142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 >> 172.21.15.64:0/588726437 conn(0x7f9da006a800 msgr2=0x7f9da0144210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 shutdown_connections 2023-11-24T06:28:01.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:01.261+0000 7f9da6def640 1 -- 172.21.15.64:0/588726437 wait complete. 2023-11-24T06:28:01.283 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:28:01.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:28:01.284 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T06:28:01.284 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:28:01.284 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:01.284 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:28:01.284 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:28:01.284 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:28:01.284 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:28:01.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:01.285 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T06:28:01.343 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:01.343 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:01.343 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:01.343 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:01.343 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:01.343 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:01.344 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:01.344 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:01.344 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T06:28:01.344 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:01.344 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:01.345 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T06:28:01.345 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:01.345 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:01.345 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:28:01.345 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:01.345 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:28:01.346 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42313sec preferred_lft 42313sec 2023-11-24T06:28:01.346 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T06:28:01.346 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:01.347 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:28:01.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:01.347 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:01.347 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:01.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:28:01.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2023-11-24T06:28:01.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2023-11-24T06:28:01.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:28:01.348 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:01.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:01.843 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:28:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T06:28:01.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:01.937 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T06:28:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T06:28:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:28:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:28:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:28:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:28:01.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:01.992 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:01.992 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:01.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:28:01.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:28:01.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:28:01.992 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:02.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:02.717 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T06:28:02.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:02.768 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-11-24T06:28:02.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:02.862 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:02.862 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:02.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:02.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T06:28:02.863 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:03.002 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T06:28:03.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:03.002 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:03.003 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:03.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T06:28:03.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:28:03.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T06:28:03.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T06:28:03.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T06:28:03.003 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:03.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:03.265 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:03.265 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:03.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T06:28:03.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:28:03.265 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:03.425 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:28:03.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:03.425 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:03.471 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:28:03.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:03.472 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:03.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T06:28:03.652 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:03.695 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T06:28:03.695 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T06:28:03.695 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T06:28:03.695 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T06:28:03.695 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T06:28:03.695 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T06:28:03.695 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T06:28:03.696 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:03.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:03.699 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T06:28:03.793 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:28:03.793 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:28:03.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T06:28:03.867 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:03.909 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T06:28:03.909 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T06:28:03.909 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T06:28:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:28:03.911 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:28:03.912 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:28:03.912 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T06:28:03.912 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:03.912 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:03.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:03.913 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T06:28:03.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T06:28:03.990+0000 7f00e3e40480 -1 init, newargv = 0x56295ab70cb0 newargc=15 2023-11-24T06:28:03.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[96156]: starting ceph client 2023-11-24T06:28:04.006 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[96156]: starting fuse 2023-11-24T06:28:05.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T06:28:05.213 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:05.277 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T06:28:05.277 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T06:28:05.277 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T06:28:05.277 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T06:28:05.278 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T06:28:05.279 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:05.280 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:05.280 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:05.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:05.281 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T06:28:05.337 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T06:28:05.337 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:28:05.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:28:05.337 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T06:28:05.338 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T06:28:05.339 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T06:28:05.525 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.96156.asok 2023-11-24T06:28:05.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:28:05.544 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.96156.asok 2023-11-24T06:28:05.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:05.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.96156.asok status 2023-11-24T06:28:05.800 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:28:05.800 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T06:28:05.800 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "96156", 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4653, 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T06:28:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4653 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1338764138 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T06:28:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1338764138 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4653 192.168.0.1:0/1338764138", 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/1338764138", 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 22, 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T06:28:05.803 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:28:05.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:28:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.039+0000 7fd293fff640 1 Processor -- start 2023-11-24T06:28:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.040+0000 7fd293fff640 1 -- start start 2023-11-24T06:28:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.042+0000 7fd293fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2940ce310 0x7fd2940ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.042+0000 7fd293fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2940d0640 0x7fd294166f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.042+0000 7fd293fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940b1430 con 0x7fd2940cd820 2023-11-24T06:28:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.042+0000 7fd293fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940d0a50 con 0x7fd2940ce310 2023-11-24T06:28:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.042+0000 7fd293fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940d0c40 con 0x7fd2940d0640 2023-11-24T06:28:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.042+0000 7fd292ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd2940cd820 0x7fd2940cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46110/0 (socket says 172.21.15.64:46110) 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.042+0000 7fd292ffd640 1 -- 172.21.15.64:0/3636038120 learned_addr learned my addr 172.21.15.64:0/3636038120 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd28bfff640 1 --2- 172.21.15.64:0/3636038120 >> v2:172.21.15.64:3300/0 conn(0x7fd2940ce310 0x7fd2940ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd2937fe640 1 --2- 172.21.15.64:0/3636038120 >> v2:172.21.15.162:3300/0 conn(0x7fd2940d0640 0x7fd294166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd2927fc640 1 -- 172.21.15.64:0/3636038120 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100968780 0 0) 0x7fd2940b1430 con 0x7fd2940cd820 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd2927fc640 1 -- 172.21.15.64:0/3636038120 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd280003520 con 0x7fd2940cd820 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd28bfff640 1 -- 172.21.15.64:0/3636038120 >> v2:172.21.15.162:3300/0 conn(0x7fd2940d0640 msgr2=0x7fd294166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd28bfff640 1 --2- 172.21.15.64:0/3636038120 >> v2:172.21.15.162:3300/0 conn(0x7fd2940d0640 0x7fd294166f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd28bfff640 1 -- 172.21.15.64:0/3636038120 >> v1:172.21.15.64:6789/0 conn(0x7fd2940cd820 legacy=0x7fd2940cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.043+0000 7fd28bfff640 1 -- 172.21.15.64:0/3636038120 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940d0f20 con 0x7fd2940ce310 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.044+0000 7fd28bfff640 1 --2- 172.21.15.64:0/3636038120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940ce310 0x7fd2940ced50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2940d00b0 tx=0x7fd27c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea57aaa319ac7ea1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.044+0000 7fd2927fc640 1 -- 172.21.15.64:0/3636038120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c02e070 con 0x7fd2940ce310 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.044+0000 7fd2927fc640 1 -- 172.21.15.64:0/3636038120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c0315a0 con 0x7fd2940ce310 2023-11-24T06:28:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.044+0000 7fd2927fc640 1 -- 172.21.15.64:0/3636038120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c031910 con 0x7fd2940ce310 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 -- 172.21.15.64:0/3636038120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940ce310 msgr2=0x7fd2940ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 --2- 172.21.15.64:0/3636038120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940ce310 0x7fd2940ced50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2940d00b0 tx=0x7fd27c02b840 comp rx=0 tx=0).stop 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 -- 172.21.15.64:0/3636038120 shutdown_connections 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 --2- 172.21.15.64:0/3636038120 >> v2:172.21.15.162:3300/0 conn(0x7fd2940d0640 0x7fd294166f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 --2- 172.21.15.64:0/3636038120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940ce310 0x7fd2940ced50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 -- 172.21.15.64:0/3636038120 >> 172.21.15.64:0/3636038120 conn(0x7fd294099a60 msgr2=0x7fd294099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 -- 172.21.15.64:0/3636038120 shutdown_connections 2023-11-24T06:28:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.045+0000 7fd293fff640 1 -- 172.21.15.64:0/3636038120 wait complete. 2023-11-24T06:28:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.047+0000 7fd293fff640 1 Processor -- start 2023-11-24T06:28:06.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.047+0000 7fd293fff640 1 -- start start 2023-11-24T06:28:06.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd293fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2940cd820 0x7fd29409e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd293fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2940ce310 0x7fd2940cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd293fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940d0640 0x7fd29406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd293fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2941f28a0 con 0x7fd2940d0640 2023-11-24T06:28:06.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd293fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940cc6b0 con 0x7fd2940ce310 2023-11-24T06:28:06.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd293fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29415ea80 con 0x7fd2940cd820 2023-11-24T06:28:06.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd2937fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940d0640 0x7fd29406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd2937fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940d0640 0x7fd29406e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47370/0 (socket says 172.21.15.64:47370) 2023-11-24T06:28:06.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.048+0000 7fd2937fe640 1 -- 172.21.15.64:0/3955542594 learned_addr learned my addr 172.21.15.64:0/3955542594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:06.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd292ffd640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2940cd820 0x7fd29409e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd28bfff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2940ce310 0x7fd2940cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd2937fe640 1 -- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2940cd820 msgr2=0x7fd29409e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd2937fe640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2940cd820 0x7fd29409e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd2937fe640 1 -- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2940ce310 msgr2=0x7fd2940cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd2937fe640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2940ce310 0x7fd2940cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd2937fe640 1 -- 172.21.15.64:0/3955542594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c009400 con 0x7fd2940d0640 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd28bfff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2940ce310 0x7fd2940cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd2937fe640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940d0640 0x7fd29406e9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28c007e40 tx=0x7fd28c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd290ff9640 1 -- 172.21.15.64:0/3955542594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c014620 con 0x7fd2940d0640 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd290ff9640 1 -- 172.21.15.64:0/3955542594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00ee90 con 0x7fd2940d0640 2023-11-24T06:28:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd290ff9640 1 -- 172.21.15.64:0/3955542594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0149e0 con 0x7fd2940d0640 2023-11-24T06:28:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.049+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29415df00 con 0x7fd2940d0640 2023-11-24T06:28:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.050+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29415e4c0 con 0x7fd2940d0640 2023-11-24T06:28:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.051+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2941f29e0 con 0x7fd2940d0640 2023-11-24T06:28:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.051+0000 7fd290ff9640 1 -- 172.21.15.64:0/3955542594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd28c014b40 con 0x7fd2940d0640 2023-11-24T06:28:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.053+0000 7fd290ff9640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd27406ae40 0x7fd27406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.053+0000 7fd290ff9640 1 -- 172.21.15.64:0/3955542594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd28c08a0d0 con 0x7fd2940d0640 2023-11-24T06:28:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.053+0000 7fd292ffd640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd27406ae40 0x7fd27406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.054+0000 7fd292ffd640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd27406ae40 0x7fd27406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd284007fd0 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.058+0000 7fd290ff9640 1 -- 172.21.15.64:0/3955542594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd28c0550e0 con 0x7fd2940d0640 2023-11-24T06:28:06.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.293+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd294067450 con 0x7fd2940d0640 2023-11-24T06:28:06.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.294+0000 7fd290ff9640 1 -- 172.21.15.64:0/3955542594 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd28c058700 con 0x7fd2940d0640 2023-11-24T06:28:06.297 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd27406ae40 msgr2=0x7fd27406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd27406ae40 0x7fd27406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd284007fd0 tx=0x7fd284009040 comp rx=0 tx=0).stop 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940d0640 msgr2=0x7fd29406e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940d0640 0x7fd29406e9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28c007e40 tx=0x7fd28c00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 shutdown_connections 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd27406ae40 0x7fd27406d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2940d0640 0x7fd29406e9c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2940ce310 0x7fd2940cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 --2- 172.21.15.64:0/3955542594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2940cd820 0x7fd29409e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 >> 172.21.15.64:0/3955542594 conn(0x7fd294099a60 msgr2=0x7fd29415cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 shutdown_connections 2023-11-24T06:28:06.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:06.297+0000 7fd293fff640 1 -- 172.21.15.64:0/3955542594 wait complete. 2023-11-24T06:28:06.318 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:28:06.319 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:28:06.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:06.319 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:06.374 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42308sec preferred_lft 42308sec 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:06.375 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 92:32:08:7d:09:ea brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::64a0:9fff:fe5c:508c/64 scope link 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 92:32:08:7d:09:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9032:8ff:fe7d:9ea/64 scope link 2023-11-24T06:28:06.376 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:06.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:06.377 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:28:06.476 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T06:28:06.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:06.480 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T06:28:06.534 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T06:28:06.535 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:28:06.535 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:28:06.535 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:28:06.535 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:28:06.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:06.535 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:06.535 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:06.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:28:06.536 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:28:06.536 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:28:06.536 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:06.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:06.738 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T06:28:06.796 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:28:06.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:06.797 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-11-24T06:28:06.852 INFO:teuthology.orchestra.run.smithi064.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T06:28:06.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:06.854 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:06.854 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:06.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:06.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T06:28:06.854 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:07.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:07.010 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:28:07.106 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 72:78:db:8c:80:c8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::7078:dbff:fe8c:80c8/64 scope link 2023-11-24T06:28:07.107 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:07.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 72:78:db:8c:80:c8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::7078:dbff:fe8c:80c8/64 scope link 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:07.214 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T06:28:07.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T06:28:07.214 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:07.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:07.482 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:28:07.482 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:28:07.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.1 up 2023-11-24T06:28:07.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:28:07.482 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:28:07.651 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-24T06:28:07.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:07.652 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:07.704 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:28:07.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:07.704 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:07.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T06:28:07.856 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:07.915 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T06:28:07.915 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:07.916 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:28:07.917 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:07.918 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:07.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:07.919 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T06:28:07.976 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T06:28:07.976 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-11-24T06:28:07.977 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-24T06:28:07.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T06:28:08.064 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:08.109 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T06:28:08.109 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T06:28:08.109 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T06:28:08.109 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T06:28:08.109 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T06:28:08.109 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T06:28:08.110 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:28:08.111 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:08.112 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:08.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:08.113 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T06:28:08.172 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:2023-11-24T06:28:08.170+0000 7f6bf2c40480 -1 init, newargv = 0x56352e649770 newargc=15 2023-11-24T06:28:08.173 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[96718]: starting ceph client 2023-11-24T06:28:08.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[96718]: starting fuse 2023-11-24T06:28:08.201 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T06:28:08.201 INFO:teuthology.orchestra.run.smithi064.stdout:42 2023-11-24T06:28:08.202 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-11-24T06:28:08.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:28:08.202 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T06:28:08.202 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T06:28:08.202 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T06:28:08.202 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T06:28:08.202 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:08.202 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:28:08.203 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:28:08.204 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T06:28:08.204 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T06:28:08.204 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T06:28:08.204 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:28:08.204 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:28:08.204 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.2")) 2023-11-24T06:28:08.204 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T06:28:08.404 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.2.96718.asok 2023-11-24T06:28:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:28:08.421 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.96718.asok 2023-11-24T06:28:08.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:08.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.96718.asok status 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "2", 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "96718", 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4671, 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4671 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 291527568 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 291527568 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4671 192.168.0.2:0/291527568", 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.2:0/291527568", 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T06:28:08.658 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2023-11-24T06:28:08.658 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 22, 2023-11-24T06:28:08.658 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:28:08.658 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T06:28:08.658 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T06:28:08.658 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:28:08.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:28:08.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.899+0000 7fb17025c640 1 Processor -- start 2023-11-24T06:28:08.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.899+0000 7fb17025c640 1 -- start start 2023-11-24T06:28:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb17025c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb16806a650 0x7fb1680ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb17025c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb16806aa30 0x7fb168071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb17025c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16813e8c0 0x7fb16813ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb17025c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1680cfa50 con 0x7fb16806a650 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb17025c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1680cee40 con 0x7fb16806aa30 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb17025c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680cf120 con 0x7fb16813e8c0 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb16dff8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb16806a650 0x7fb1680ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb16dff8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb16806a650 0x7fb1680ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47384/0 (socket says 172.21.15.64:47384) 2023-11-24T06:28:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb16dff8640 1 -- 172.21.15.64:0/129290908 learned_addr learned my addr 172.21.15.64:0/129290908 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.901+0000 7fb16e7f9640 1 --2- 172.21.15.64:0/129290908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16813e8c0 0x7fb16813ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.902+0000 7fb16d7f7640 1 --2- 172.21.15.64:0/129290908 >> v2:172.21.15.162:3300/0 conn(0x7fb16806aa30 0x7fb168071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.902+0000 7fb16dff8640 1 -- 172.21.15.64:0/129290908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16813e8c0 msgr2=0x7fb16813ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.902+0000 7fb16dff8640 1 --2- 172.21.15.64:0/129290908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16813e8c0 0x7fb16813ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.902+0000 7fb16dff8640 1 -- 172.21.15.64:0/129290908 >> v2:172.21.15.162:3300/0 conn(0x7fb16806aa30 msgr2=0x7fb168071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.902+0000 7fb16dff8640 1 --2- 172.21.15.64:0/129290908 >> v2:172.21.15.162:3300/0 conn(0x7fb16806aa30 0x7fb168071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.902+0000 7fb16dff8640 1 -- 172.21.15.64:0/129290908 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680cf400 con 0x7fb16806a650 2023-11-24T06:28:08.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.903+0000 7fb16dff8640 1 --2- 172.21.15.64:0/129290908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb1680ce7f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb1680cd9e0 tx=0x7fb16402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4aa43dc9a1b138 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.903+0000 7fb16cff6640 1 -- 172.21.15.64:0/129290908 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164031870 con 0x7fb16806a650 2023-11-24T06:28:08.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb16cff6640 1 -- 172.21.15.64:0/129290908 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1640319d0 con 0x7fb16806a650 2023-11-24T06:28:08.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb16cff6640 1 -- 172.21.15.64:0/129290908 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164031d60 con 0x7fb16806a650 2023-11-24T06:28:08.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 -- 172.21.15.64:0/129290908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 msgr2=0x7fb1680ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 --2- 172.21.15.64:0/129290908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb1680ce7f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb1680cd9e0 tx=0x7fb16402b910 comp rx=0 tx=0).stop 2023-11-24T06:28:08.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 -- 172.21.15.64:0/129290908 shutdown_connections 2023-11-24T06:28:08.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 --2- 172.21.15.64:0/129290908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16813e8c0 0x7fb16813ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 --2- 172.21.15.64:0/129290908 >> v2:172.21.15.162:3300/0 conn(0x7fb16806aa30 0x7fb168071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 --2- 172.21.15.64:0/129290908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb1680ce7f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 -- 172.21.15.64:0/129290908 >> 172.21.15.64:0/129290908 conn(0x7fb1680b1bf0 msgr2=0x7fb1680b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:08.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 -- 172.21.15.64:0/129290908 shutdown_connections 2023-11-24T06:28:08.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.904+0000 7fb17025c640 1 -- 172.21.15.64:0/129290908 wait complete. 2023-11-24T06:28:08.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.906+0000 7fb17025c640 1 Processor -- start 2023-11-24T06:28:08.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.906+0000 7fb17025c640 1 -- start start 2023-11-24T06:28:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.907+0000 7fb17025c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb16813fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.907+0000 7fb17025c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16806aa30 0x7fb16815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.907+0000 7fb17025c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16813e8c0 0x7fb1681f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.907+0000 7fb17025c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168152a00 con 0x7fb16806a650 2023-11-24T06:28:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.907+0000 7fb17025c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168152ce0 con 0x7fb16813e8c0 2023-11-24T06:28:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.907+0000 7fb17025c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb168152fc0 con 0x7fb16806aa30 2023-11-24T06:28:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb16813fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb16813fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47390/0 (socket says 172.21.15.64:47390) 2023-11-24T06:28:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 -- 172.21.15.64:0/3474100035 learned_addr learned my addr 172.21.15.64:0/3474100035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16e7f9640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16813e8c0 0x7fb1681f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16d7f7640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16806aa30 0x7fb16815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 -- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16806aa30 msgr2=0x7fb16815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16806aa30 0x7fb16815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 -- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16813e8c0 msgr2=0x7fb1681f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16813e8c0 0x7fb1681f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.908+0000 7fb16dff8640 1 -- 172.21.15.64:0/3474100035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1640094d0 con 0x7fb16806a650 2023-11-24T06:28:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.909+0000 7fb16dff8640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb16813fa40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb164000c00 tx=0x7fb1640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.909+0000 7fb15e7fc640 1 -- 172.21.15.64:0/3474100035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16403abe0 con 0x7fb16806a650 2023-11-24T06:28:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.909+0000 7fb15e7fc640 1 -- 172.21.15.64:0/3474100035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16403ad40 con 0x7fb16806a650 2023-11-24T06:28:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.909+0000 7fb15e7fc640 1 -- 172.21.15.64:0/3474100035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164044610 con 0x7fb16806a650 2023-11-24T06:28:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.909+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1681532a0 con 0x7fb16806a650 2023-11-24T06:28:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.909+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb168152620 con 0x7fb16806a650 2023-11-24T06:28:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.910+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1680675d0 con 0x7fb16806a650 2023-11-24T06:28:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.911+0000 7fb15e7fc640 1 -- 172.21.15.64:0/3474100035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb16404d450 con 0x7fb16806a650 2023-11-24T06:28:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.912+0000 7fb15e7fc640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb14c06ae40 0x7fb14c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.912+0000 7fb15e7fc640 1 -- 172.21.15.64:0/3474100035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb16402d040 con 0x7fb16806a650 2023-11-24T06:28:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.913+0000 7fb16d7f7640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb14c06ae40 0x7fb14c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.913+0000 7fb16d7f7640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb14c06ae40 0x7fb14c06d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb1681f6220 tx=0x7fb158008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:08.917+0000 7fb15e7fc640 1 -- 172.21.15.64:0/3474100035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb164076d00 con 0x7fb16806a650 2023-11-24T06:28:09.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.152+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb16813ddc0 con 0x7fb16806a650 2023-11-24T06:28:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.153+0000 7fb15e7fc640 1 -- 172.21.15.64:0/3474100035 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb164035020 con 0x7fb16806a650 2023-11-24T06:28:09.158 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.155+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb14c06ae40 msgr2=0x7fb14c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.155+0000 7fb17025c640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb14c06ae40 0x7fb14c06d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb1681f6220 tx=0x7fb158008040 comp rx=0 tx=0).stop 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.155+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 msgr2=0x7fb16813fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.155+0000 7fb17025c640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb16813fa40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb164000c00 tx=0x7fb1640046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 shutdown_connections 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb14c06ae40 0x7fb14c06d300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16813e8c0 0x7fb1681f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16806aa30 0x7fb16815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 --2- 172.21.15.64:0/3474100035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16806a650 0x7fb16813fa40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 >> 172.21.15.64:0/3474100035 conn(0x7fb1680b1bf0 msgr2=0x7fb168143ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 shutdown_connections 2023-11-24T06:28:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:09.156+0000 7fb17025c640 1 -- 172.21.15.64:0/3474100035 wait complete. 2023-11-24T06:28:09.180 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:28:09.181 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:28:09.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:09.181 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2023-11-24T06:28:09.244 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:09.245 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:09.245 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:09.245 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:09.245 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:09.245 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:09.245 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:09.245 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:28:09.246 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:09.247 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:28:09.247 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42326sec preferred_lft 42326sec 2023-11-24T06:28:09.247 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2023-11-24T06:28:09.247 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:09.247 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:28:09.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:09.248 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:09.248 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:09.248 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:28:09.248 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr flush dev ceph-brx 2023-11-24T06:28:09.248 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set ceph-brx up 2023-11-24T06:28:09.248 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:28:09.248 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:09.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:09.735 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:28:09.813 INFO:teuthology.orchestra.run.smithi162.stdout:1 2023-11-24T06:28:09.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:09.818 DEBUG:teuthology.orchestra.run.smithi162:> route 2023-11-24T06:28:09.869 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2023-11-24T06:28:09.869 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:28:09.869 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:28:09.869 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:28:09.869 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:28:09.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:09.870 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:09.870 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:09.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:28:09.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:28:09.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:28:09.870 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:10.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:10.623 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-11-24T06:28:10.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:10.681 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2023-11-24T06:28:10.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:10.734 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:10.735 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:10.735 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:10.735 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T06:28:10.735 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:10.879 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T06:28:10.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T06:28:10.880 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:11.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:11.168 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:11.168 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:11.168 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2023-11-24T06:28:11.168 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:28:11.169 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:11.336 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:28:11.336 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:11.336 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:11.380 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:28:11.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:11.380 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:11.477 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-11-24T06:28:11.572 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:11.632 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-11-24T06:28:11.632 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-11-24T06:28:11.632 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-11-24T06:28:11.632 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-11-24T06:28:11.632 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-11-24T06:28:11.632 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-11-24T06:28:11.633 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:11.634 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:11.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:11.635 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-11-24T06:28:11.682 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:28:11.682 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:28:11.731 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-11-24T06:28:11.814 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:11.858 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-11-24T06:28:11.858 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-11-24T06:28:11.858 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-11-24T06:28:11.858 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-11-24T06:28:11.858 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-11-24T06:28:11.858 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:11.859 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:28:11.860 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-11-24T06:28:11.861 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:11.861 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:11.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:11.862 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-11-24T06:28:11.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:2023-11-24T06:28:11.953+0000 7f2d22c10480 -1 init, newargv = 0x5641dea89970 newargc=15 2023-11-24T06:28:11.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[94328]: starting ceph client 2023-11-24T06:28:11.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[94328]: starting fuse 2023-11-24T06:28:13.051 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-11-24T06:28:13.160 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:13.215 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-11-24T06:28:13.216 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:13.217 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-clean.service 2023-11-24T06:28:13.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:13.222 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-11-24T06:28:13.267 INFO:teuthology.orchestra.run.smithi162.stdout:41 2023-11-24T06:28:13.268 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:28:13.268 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:28:13.268 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2023-11-24T06:28:13.268 DEBUG:teuthology.orchestra.run.smithi162:> import re 2023-11-24T06:28:13.268 DEBUG:teuthology.orchestra.run.smithi162:> import os 2023-11-24T06:28:13.268 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2023-11-24T06:28:13.268 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:13.269 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2023-11-24T06:28:13.269 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:28:13.269 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2023-11-24T06:28:13.269 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:28:13.269 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2023-11-24T06:28:13.270 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2023-11-24T06:28:13.271 DEBUG:teuthology.orchestra.run.smithi162:> return f 2023-11-24T06:28:13.271 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:28:13.271 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:13.271 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.1")) 2023-11-24T06:28:13.271 DEBUG:teuthology.orchestra.run.smithi162:> ' 2023-11-24T06:28:13.496 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.1.94328.asok 2023-11-24T06:28:13.500 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:28:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:28:13.514 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94328.asok 2023-11-24T06:28:13.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:13.514 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94328.asok status 2023-11-24T06:28:13.754 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-11-24T06:28:13.754 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "1", 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "94328", 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2023-11-24T06:28:13.755 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 4519, 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 4519 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:13.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 1517161461 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 1517161461 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.4519 192.168.0.1:0/1517161461", 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.1:0/1517161461", 2023-11-24T06:28:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2023-11-24T06:28:13.758 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 8, 2023-11-24T06:28:13.758 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 22, 2023-11-24T06:28:13.758 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:28:13.758 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2023-11-24T06:28:13.758 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2023-11-24T06:28:13.758 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-11-24T06:28:13.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:28:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:13.999+0000 7f449359e640 1 Processor -- start 2023-11-24T06:28:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.000+0000 7f449359e640 1 -- start start 2023-11-24T06:28:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.001+0000 7f449359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4494160920 0x7f44940b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.001+0000 7f449359e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f44940cfe70 0x7f4494161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.001+0000 7f449359e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44940d0a10 con 0x7f44940cf9f0 2023-11-24T06:28:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.001+0000 7f449359e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44940d1f50 con 0x7f44940cfe70 2023-11-24T06:28:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.001+0000 7f449359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44940cf080 con 0x7f4494160920 2023-11-24T06:28:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.002+0000 7f449259c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4494160920 0x7f44940b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.002+0000 7f4492d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f44940cfe70 0x7f4494161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.002+0000 7f449259c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4494160920 0x7f44940b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:43290/0 (socket says 172.21.15.64:43290) 2023-11-24T06:28:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.002+0000 7f4492d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f44940cfe70 0x7f4494161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:37374/0 (socket says 172.21.15.64:37374) 2023-11-24T06:28:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.002+0000 7f449259c640 1 -- 172.21.15.64:0/3124483503 learned_addr learned my addr 172.21.15.64:0/3124483503 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.002+0000 7f449259c640 1 -- 172.21.15.64:0/3124483503 >> v2:172.21.15.162:3300/0 conn(0x7f44940cfe70 msgr2=0x7f4494161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.002+0000 7f449259c640 1 --2- 172.21.15.64:0/3124483503 >> v2:172.21.15.162:3300/0 conn(0x7f44940cfe70 0x7f4494161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.003+0000 7f449259c640 1 -- 172.21.15.64:0/3124483503 >> v1:172.21.15.162:6789/0 conn(0x7f44940cf9f0 legacy=0x7f4494161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.003+0000 7f449259c640 1 -- 172.21.15.64:0/3124483503 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940c6be0 con 0x7f4494160920 2023-11-24T06:28:14.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.004+0000 7f449259c640 1 --2- 172.21.15.64:0/3124483503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4494160920 0x7f44940b3f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4484009e50 tx=0x7f448402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5cd31230d8f3ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.005+0000 7f449159a640 1 -- 172.21.15.64:0/3124483503 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448402cb00 con 0x7f4494160920 2023-11-24T06:28:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.005+0000 7f449159a640 1 -- 172.21.15.64:0/3124483503 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448402cc60 con 0x7f4494160920 2023-11-24T06:28:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.005+0000 7f449159a640 1 -- 172.21.15.64:0/3124483503 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840343f0 con 0x7f4494160920 2023-11-24T06:28:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 -- 172.21.15.64:0/3124483503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4494160920 msgr2=0x7f44940b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 --2- 172.21.15.64:0/3124483503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4494160920 0x7f44940b3f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4484009e50 tx=0x7f448402bf60 comp rx=0 tx=0).stop 2023-11-24T06:28:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 -- 172.21.15.64:0/3124483503 shutdown_connections 2023-11-24T06:28:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 --2- 172.21.15.64:0/3124483503 >> v2:172.21.15.162:3300/0 conn(0x7f44940cfe70 0x7f4494161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 --2- 172.21.15.64:0/3124483503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4494160920 0x7f44940b3f30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 -- 172.21.15.64:0/3124483503 >> 172.21.15.64:0/3124483503 conn(0x7f44940d3090 msgr2=0x7f44940d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 -- 172.21.15.64:0/3124483503 shutdown_connections 2023-11-24T06:28:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.006+0000 7f449359e640 1 -- 172.21.15.64:0/3124483503 wait complete. 2023-11-24T06:28:14.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.008+0000 7f449359e640 1 Processor -- start 2023-11-24T06:28:14.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.009+0000 7f449359e640 1 -- start start 2023-11-24T06:28:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f449359e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44940cf9f0 0x7f44940b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f449359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44940cfe70 0x7f44940ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f449359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4494160920 0x7f449415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f449359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449406cbb0 con 0x7f4494160920 2023-11-24T06:28:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f449359e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44940cca60 con 0x7f44940cf9f0 2023-11-24T06:28:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f449359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44941f3a70 con 0x7f44940cfe70 2023-11-24T06:28:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f4492d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4494160920 0x7f449415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.010+0000 7f4492d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4494160920 0x7f449415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41598/0 (socket says 172.21.15.64:41598) 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4492d9d640 1 -- 172.21.15.64:0/856645308 learned_addr learned my addr 172.21.15.64:0/856645308 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4491d9b640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44940cfe70 0x7f44940ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f449259c640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44940cf9f0 0x7f44940b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4492d9d640 1 -- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44940cfe70 msgr2=0x7f44940ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4492d9d640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44940cfe70 0x7f44940ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4492d9d640 1 -- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44940cf9f0 msgr2=0x7f44940b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4492d9d640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44940cf9f0 0x7f44940b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4492d9d640 1 -- 172.21.15.64:0/856645308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484009b30 con 0x7f4494160920 2023-11-24T06:28:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4491d9b640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44940cfe70 0x7f44940ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.011+0000 7f4492d9d640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4494160920 0x7f449415aff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f448c00e930 tx=0x7f448c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.012+0000 7f448affd640 1 -- 172.21.15.64:0/856645308 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c009690 con 0x7f4494160920 2023-11-24T06:28:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.012+0000 7f448affd640 1 -- 172.21.15.64:0/856645308 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c00c980 con 0x7f4494160920 2023-11-24T06:28:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.012+0000 7f448affd640 1 -- 172.21.15.64:0/856645308 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00cd30 con 0x7f4494160920 2023-11-24T06:28:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.012+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44941f3d50 con 0x7f4494160920 2023-11-24T06:28:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.012+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44940cc6f0 con 0x7f4494160920 2023-11-24T06:28:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.013+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44940cc270 con 0x7f4494160920 2023-11-24T06:28:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.014+0000 7f448affd640 1 -- 172.21.15.64:0/856645308 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f448c013470 con 0x7f4494160920 2023-11-24T06:28:14.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.015+0000 7f448affd640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f447406ae40 0x7f447406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.015+0000 7f448affd640 1 -- 172.21.15.64:0/856645308 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f448c089820 con 0x7f4494160920 2023-11-24T06:28:14.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.015+0000 7f449259c640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f447406ae40 0x7f447406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.016+0000 7f449259c640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f447406ae40 0x7f447406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4484035000 tx=0x7f4484030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.020+0000 7f448affd640 1 -- 172.21.15.64:0/856645308 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44940cc270 con 0x7f4494160920 2023-11-24T06:28:14.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.256+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f44940cc270 con 0x7f4494160920 2023-11-24T06:28:14.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.256+0000 7f448affd640 1 -- 172.21.15.64:0/856645308 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f44940cc270 con 0x7f4494160920 2023-11-24T06:28:14.262 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:28:14.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f447406ae40 msgr2=0x7f447406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f447406ae40 0x7f447406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4484035000 tx=0x7f4484030000 comp rx=0 tx=0).stop 2023-11-24T06:28:14.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4494160920 msgr2=0x7f449415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4494160920 0x7f449415aff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f448c00e930 tx=0x7f448c00edf0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 shutdown_connections 2023-11-24T06:28:14.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f447406ae40 0x7f447406d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4494160920 0x7f449415aff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44940cfe70 0x7f44940ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 --2- 172.21.15.64:0/856645308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44940cf9f0 0x7f44940b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.259+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 >> 172.21.15.64:0/856645308 conn(0x7f44940d3090 msgr2=0x7f44940b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:14.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.260+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 shutdown_connections 2023-11-24T06:28:14.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:14.260+0000 7f449359e640 1 -- 172.21.15.64:0/856645308 wait complete. 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:28:14.280 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:28:14.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.281 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2023-11-24T06:28:14.341 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42321sec preferred_lft 42321sec 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:14.342 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether ea:a0:b9:23:79:12 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::5c20:50ff:fe3d:265e/64 scope link 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether ea:a0:b9:23:79:12 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::e8a0:b9ff:fe23:7912/64 scope link 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:14.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.344 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:28:14.437 INFO:teuthology.orchestra.run.smithi162.stdout:1 2023-11-24T06:28:14.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.441 DEBUG:teuthology.orchestra.run.smithi162:> route 2023-11-24T06:28:14.495 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2023-11-24T06:28:14.495 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:28:14.495 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:28:14.495 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:28:14.495 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:28:14.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.496 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:14.496 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:14.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:28:14.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:28:14.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:28:14.496 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:14.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.673 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-11-24T06:28:14.721 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:28:14.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.722 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2023-11-24T06:28:14.821 INFO:teuthology.orchestra.run.smithi162.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T06:28:14.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.822 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:14.822 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:14.822 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:14.823 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T06:28:14.823 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:14.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:14.966 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:28:15.063 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 1e:54:8c:d5:2b:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::1c54:8cff:fed5:2bb7/64 scope link 2023-11-24T06:28:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:15.068 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 1e:54:8c:d5:2b:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::1c54:8cff:fed5:2bb7/64 scope link 2023-11-24T06:28:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:15.150 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T06:28:15.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:15.150 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:15.150 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:15.150 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T06:28:15.150 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:28:15.151 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T06:28:15.151 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T06:28:15.151 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T06:28:15.151 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:15.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:15.406 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:28:15.406 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:28:15.406 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.1 up 2023-11-24T06:28:15.406 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:28:15.406 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:28:15.576 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-24T06:28:15.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:15.577 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:15.627 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:28:15.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:15.628 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:15.675 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-11-24T06:28:15.816 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:15.873 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-11-24T06:28:15.873 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-11-24T06:28:15.873 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-11-24T06:28:15.873 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-11-24T06:28:15.873 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-11-24T06:28:15.873 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-11-24T06:28:15.874 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:28:15.875 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-11-24T06:28:15.876 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:15.876 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:15.876 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:15.876 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:15.876 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:15.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:15.877 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-11-24T06:28:15.929 INFO:teuthology.orchestra.run.smithi162.stdout:41 2023-11-24T06:28:15.929 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-24T06:28:15.929 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-24T06:28:15.937 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-11-24T06:28:16.023 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:16.066 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-11-24T06:28:16.066 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-11-24T06:28:16.066 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-11-24T06:28:16.066 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-11-24T06:28:16.067 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:28:16.068 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:28:16.069 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:28:16.069 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-11-24T06:28:16.069 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:16.069 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:16.069 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:16.069 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:16.069 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:16.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:16.070 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-11-24T06:28:16.133 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:2023-11-24T06:28:16.129+0000 7f5b53cf6480 -1 init, newargv = 0x55fa0d809970 newargc=15 2023-11-24T06:28:16.133 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:ceph-fuse[94845]: starting ceph client 2023-11-24T06:28:16.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:ceph-fuse[94845]: starting fuse 2023-11-24T06:28:16.158 INFO:teuthology.orchestra.run.smithi162.stdout:41 2023-11-24T06:28:16.158 INFO:teuthology.orchestra.run.smithi162.stdout:42 2023-11-24T06:28:16.158 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2023-11-24T06:28:16.158 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> import re 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> import os 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2023-11-24T06:28:16.159 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> return f 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.3")) 2023-11-24T06:28:16.160 DEBUG:teuthology.orchestra.run.smithi162:> ' 2023-11-24T06:28:16.359 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.3.94845.asok 2023-11-24T06:28:16.363 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:28:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:28:16.376 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.94845.asok 2023-11-24T06:28:16.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:16.376 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.94845.asok status 2023-11-24T06:28:16.616 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-11-24T06:28:16.616 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2023-11-24T06:28:16.616 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:28:16.616 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:28:16.617 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "3", 2023-11-24T06:28:16.617 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2023-11-24T06:28:16.617 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-24T06:28:16.617 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "94845", 2023-11-24T06:28:16.617 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2023-11-24T06:28:16.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:16.617 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2023-11-24T06:28:16.618 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2023-11-24T06:28:16.618 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 4463, 2023-11-24T06:28:16.618 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2023-11-24T06:28:16.618 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2023-11-24T06:28:16.618 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 4463 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 4114602688 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:28:16.619 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 4114602688 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.4463 192.168.0.2:0/4114602688", 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.2:0/4114602688", 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 8, 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 22, 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2023-11-24T06:28:16.620 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-11-24T06:28:16.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:16.632 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T06:28:16.632 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:16.699 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T06:28:16.699 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:16.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:16.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:16.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:16.808 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T06:28:16.808 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:16.875 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T06:28:16.875 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:16.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:16.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:16.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:16.985 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2023-11-24T06:28:16.985 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:17.046 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2023-11-24T06:28:17.046 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:17.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:17.047 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:17.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:17.144 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2023-11-24T06:28:17.144 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:17.197 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2023-11-24T06:28:17.197 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:17.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:17.197 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:17.294 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-24T06:28:17.306 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T06:28:17.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:17.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.592+0000 7f40b2e4b640 1 Processor -- start 2023-11-24T06:28:17.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.592+0000 7f40b2e4b640 1 -- start start 2023-11-24T06:28:17.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.593+0000 7f40b2e4b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40ac0cc500 0x7f40ac0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.593+0000 7f40b2e4b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ac06d160 0x7f40ac06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b2e4b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac070e50 con 0x7f40ac143c40 2023-11-24T06:28:17.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b2e4b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0723c0 con 0x7f40ac0cc500 2023-11-24T06:28:17.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b2e4b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0cced0 con 0x7f40ac06d160 2023-11-24T06:28:17.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b1e49640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f40ac143c40 0x7f40ac071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58184/0 (socket says 172.21.15.64:58184) 2023-11-24T06:28:17.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b1e49640 1 -- 172.21.15.64:0/1561489884 learned_addr learned my addr 172.21.15.64:0/1561489884 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:17.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877548892 0 0) 0x7f40ac070e50 con 0x7f40ac143c40 2023-11-24T06:28:17.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b264a640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ac06d160 0x7f40ac06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b1648640 1 --2- 172.21.15.64:0/1561489884 >> v2:172.21.15.162:3300/0 conn(0x7f40ac0cc500 0x7f40ac0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.594+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4098003520 con 0x7f40ac143c40 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029205355 0 0) 0x7f4098003520 con 0x7f40ac143c40 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ac06d160 msgr2=0x7f40ac06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ac06d160 0x7f40ac06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 >> v2:172.21.15.162:3300/0 conn(0x7f40ac0cc500 msgr2=0x7f40ac0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 --2- 172.21.15.64:0/1561489884 >> v2:172.21.15.162:3300/0 conn(0x7f40ac0cc500 0x7f40ac0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac144020 con 0x7f40ac143c40 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f40a8001690 con 0x7f40ac143c40 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 >> v1:172.21.15.64:6789/0 conn(0x7f40ac143c40 legacy=0x7f40ac071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4098003970 0x7f4098003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40980047c0 0x7f4098004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ac06d160 0x7f4098005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0723c0 con 0x7f40980047c0 2023-11-24T06:28:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0cced0 con 0x7f4098003970 2023-11-24T06:28:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409800b9b0 con 0x7f40ac06d160 2023-11-24T06:28:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.595+0000 7f40b1648640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40980047c0 0x7f4098004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.596+0000 7f40b1e49640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4098003970 0x7f4098003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.596+0000 7f40b1648640 1 -- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ac06d160 msgr2=0x7f4098005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.596+0000 7f40b1648640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ac06d160 0x7f4098005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.596+0000 7f40b1648640 1 -- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4098003970 msgr2=0x7f4098003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.596+0000 7f40b1648640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4098003970 0x7f4098003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.596+0000 7f40b1648640 1 -- 172.21.15.64:0/1561489884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098003520 con 0x7f40980047c0 2023-11-24T06:28:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.596+0000 7f40b1e49640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4098003970 0x7f4098003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:17.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.597+0000 7f40b1648640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40980047c0 0x7f4098004c40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f409c009ac0 tx=0x7f40980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99cb5aa384642e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.597+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c02c6d0 con 0x7f40980047c0 2023-11-24T06:28:17.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.597+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c02c830 con 0x7f40980047c0 2023-11-24T06:28:17.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.597+0000 7f40b0e47640 1 -- 172.21.15.64:0/1561489884 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c02cbc0 con 0x7f40980047c0 2023-11-24T06:28:17.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.598+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1561489884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40980047c0 msgr2=0x7f4098004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.598+0000 7f40b2e4b640 1 --2- 172.21.15.64:0/1561489884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40980047c0 0x7f4098004c40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f409c009ac0 tx=0x7f40980057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.598+0000 7f40b264a640 1 -- 172.21.15.64:0/1561489884 reap_dead start 2023-11-24T06:28:17.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.598+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1561489884 shutdown_connections 2023-11-24T06:28:17.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.598+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1561489884 >> 172.21.15.64:0/1561489884 conn(0x7f40ac06a800 msgr2=0x7f40ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.598+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1561489884 shutdown_connections 2023-11-24T06:28:17.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.598+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1561489884 wait complete. 2023-11-24T06:28:17.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.600+0000 7f40b2e4b640 1 Processor -- start 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.600+0000 7f40b2e4b640 1 -- start start 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b2e4b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40980047c0 0x7f40ac15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b2e4b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ac1f0d00 0x7f40ac142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b2e4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ac1f1180 0x7f40ac142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b2e4b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac13c9a0 con 0x7f40ac1f1180 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b2e4b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac1f40d0 con 0x7f40ac1f0d00 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b2e4b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0ac880 con 0x7f40980047c0 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b264a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ac1f1180 0x7f40ac142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b264a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ac1f1180 0x7f40ac142dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41626/0 (socket says 172.21.15.64:41626) 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.601+0000 7f40b264a640 1 -- 172.21.15.64:0/1368567911 learned_addr learned my addr 172.21.15.64:0/1368567911 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b1648640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ac1f0d00 0x7f40ac142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b1e49640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40980047c0 0x7f40ac15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b264a640 1 -- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40980047c0 msgr2=0x7f40ac15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b264a640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40980047c0 0x7f40ac15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b264a640 1 -- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ac1f0d00 msgr2=0x7f40ac142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b264a640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ac1f0d00 0x7f40ac142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b264a640 1 -- 172.21.15.64:0/1368567911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0abc70 con 0x7f40ac1f1180 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b1648640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ac1f0d00 0x7f40ac142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.602+0000 7f40b264a640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ac1f1180 0x7f40ac142dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0cc3f0 tx=0x7f40a400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.603+0000 7f40a27fc640 1 -- 172.21.15.64:0/1368567911 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4010040 con 0x7f40ac1f1180 2023-11-24T06:28:17.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.603+0000 7f40a27fc640 1 -- 172.21.15.64:0/1368567911 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a400bb10 con 0x7f40ac1f1180 2023-11-24T06:28:17.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.603+0000 7f40a27fc640 1 -- 172.21.15.64:0/1368567911 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a400bea0 con 0x7f40ac1f1180 2023-11-24T06:28:17.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.603+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac0abf50 con 0x7f40ac1f1180 2023-11-24T06:28:17.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.603+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac0ac510 con 0x7f40ac1f1180 2023-11-24T06:28:17.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.604+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac0aca10 con 0x7f40ac1f1180 2023-11-24T06:28:17.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.605+0000 7f40a27fc640 1 -- 172.21.15.64:0/1368567911 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40a401f460 con 0x7f40ac1f1180 2023-11-24T06:28:17.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.606+0000 7f40a27fc640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f409006ae40 0x7f409006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.606+0000 7f40a27fc640 1 -- 172.21.15.64:0/1368567911 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40a40899e0 con 0x7f40ac1f1180 2023-11-24T06:28:17.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.606+0000 7f40b1e49640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f409006ae40 0x7f409006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.607+0000 7f40b1e49640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f409006ae40 0x7f409006d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f40a8001eb0 tx=0x7f40a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.611+0000 7f40a27fc640 1 -- 172.21.15.64:0/1368567911 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40a4054a90 con 0x7f40ac1f1180 2023-11-24T06:28:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.808+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40ac067450 con 0x7f40ac1f1180 2023-11-24T06:28:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.811+0000 7f40a27fc640 1 -- 172.21.15.64:0/1368567911 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f40a40580b0 con 0x7f40ac1f1180 2023-11-24T06:28:17.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:17.814 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.813+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f409006ae40 msgr2=0x7f409006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f409006ae40 0x7f409006d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f40a8001eb0 tx=0x7f40a8009040 comp rx=0 tx=0).stop 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ac1f1180 msgr2=0x7f40ac142dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ac1f1180 0x7f40ac142dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0cc3f0 tx=0x7f40a400b480 comp rx=0 tx=0).stop 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 shutdown_connections 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f409006ae40 0x7f409006d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ac1f1180 0x7f40ac142dc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ac1f0d00 0x7f40ac142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 --2- 172.21.15.64:0/1368567911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40980047c0 0x7f40ac15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 >> 172.21.15.64:0/1368567911 conn(0x7f40ac06a800 msgr2=0x7f40ac071290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 shutdown_connections 2023-11-24T06:28:17.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:17.814+0000 7f40b2e4b640 1 -- 172.21.15.64:0/1368567911 wait complete. 2023-11-24T06:28:17.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:17.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:18.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.063+0000 7fa11f41a640 1 Processor -- start 2023-11-24T06:28:18.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.064+0000 7fa11f41a640 1 -- start start 2023-11-24T06:28:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.065+0000 7fa11f41a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa11815b320 0x7fa1180ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.065+0000 7fa11f41a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1180cc730 con 0x7fa1180ccbc0 2023-11-24T06:28:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11f41a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1180cbe00 con 0x7fa1180ce5a0 2023-11-24T06:28:18.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11f41a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1180cc0e0 con 0x7fa11815b320 2023-11-24T06:28:18.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11dc17640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa1180ccbc0 0x7fa11815b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58200/0 (socket says 172.21.15.64:58200) 2023-11-24T06:28:18.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11e418640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa11815b320 0x7fa1180ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11dc17640 1 -- 172.21.15.64:0/4009367076 learned_addr learned my addr 172.21.15.64:0/4009367076 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:18.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11d416640 1 -- 172.21.15.64:0/4009367076 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248673047 0 0) 0x7fa1180cc730 con 0x7fa1180ccbc0 2023-11-24T06:28:18.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11d416640 1 -- 172.21.15.64:0/4009367076 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa104003520 con 0x7fa1180ccbc0 2023-11-24T06:28:18.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11e418640 1 -- 172.21.15.64:0/4009367076 >> v1:172.21.15.162:6789/0 conn(0x7fa1180ce5a0 legacy=0x7fa1180ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11e418640 1 -- 172.21.15.64:0/4009367076 >> v1:172.21.15.64:6789/0 conn(0x7fa1180ccbc0 legacy=0x7fa11815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.066+0000 7fa11e418640 1 -- 172.21.15.64:0/4009367076 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180cc3c0 con 0x7fa11815b320 2023-11-24T06:28:18.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.068+0000 7fa11e418640 1 --2- 172.21.15.64:0/4009367076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11815b320 0x7fa1180ce060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa114009d90 tx=0x7fa11402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6708138c6a583e0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.068+0000 7fa11d416640 1 -- 172.21.15.64:0/4009367076 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11402cc60 con 0x7fa11815b320 2023-11-24T06:28:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.068+0000 7fa11d416640 1 -- 172.21.15.64:0/4009367076 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11402cdc0 con 0x7fa11815b320 2023-11-24T06:28:18.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.068+0000 7fa11d416640 1 -- 172.21.15.64:0/4009367076 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114034660 con 0x7fa11815b320 2023-11-24T06:28:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.068+0000 7fa11f41a640 1 -- 172.21.15.64:0/4009367076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11815b320 msgr2=0x7fa1180ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.069+0000 7fa11f41a640 1 --2- 172.21.15.64:0/4009367076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11815b320 0x7fa1180ce060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa114009d90 tx=0x7fa11402beb0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.069+0000 7fa11f41a640 1 -- 172.21.15.64:0/4009367076 shutdown_connections 2023-11-24T06:28:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.069+0000 7fa11f41a640 1 --2- 172.21.15.64:0/4009367076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11815b320 0x7fa1180ce060 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.069+0000 7fa11f41a640 1 -- 172.21.15.64:0/4009367076 >> 172.21.15.64:0/4009367076 conn(0x7fa1180b2a60 msgr2=0x7fa1180b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.069+0000 7fa11f41a640 1 -- 172.21.15.64:0/4009367076 shutdown_connections 2023-11-24T06:28:18.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.069+0000 7fa11f41a640 1 -- 172.21.15.64:0/4009367076 wait complete. 2023-11-24T06:28:18.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.071+0000 7fa11f41a640 1 Processor -- start 2023-11-24T06:28:18.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.071+0000 7fa11f41a640 1 -- start start 2023-11-24T06:28:18.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11f41a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180ccbc0 0x7fa118147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11f41a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180ce5a0 0x7fa118158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11f41a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa11815b320 0x7fa1180ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11f41a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1181477c0 con 0x7fa1180ccbc0 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11f41a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1181582e0 con 0x7fa1180ce5a0 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11f41a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1181576d0 con 0x7fa11815b320 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11e418640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180ccbc0 0x7fa118147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11e418640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180ccbc0 0x7fa118147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41642/0 (socket says 172.21.15.64:41642) 2023-11-24T06:28:18.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.072+0000 7fa11e418640 1 -- 172.21.15.64:0/2559530798 learned_addr learned my addr 172.21.15.64:0/2559530798 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:18.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11dc17640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180ce5a0 0x7fa118158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11ec19640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa11815b320 0x7fa1180ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11e418640 1 -- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa11815b320 msgr2=0x7fa1180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11e418640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa11815b320 0x7fa1180ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11e418640 1 -- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180ce5a0 msgr2=0x7fa118158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11e418640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180ce5a0 0x7fa118158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11e418640 1 -- 172.21.15.64:0/2559530798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114009a70 con 0x7fa1180ccbc0 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11ec19640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa11815b320 0x7fa1180ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11dc17640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180ce5a0 0x7fa118158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11e418640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180ccbc0 0x7fa118147280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa114037040 tx=0x7fa114004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa10effd640 1 -- 172.21.15.64:0/2559530798 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114040650 con 0x7fa1180ccbc0 2023-11-24T06:28:18.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa10effd640 1 -- 172.21.15.64:0/2559530798 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1140407f0 con 0x7fa1180ccbc0 2023-11-24T06:28:18.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1181579b0 con 0x7fa1180ccbc0 2023-11-24T06:28:18.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.073+0000 7fa10effd640 1 -- 172.21.15.64:0/2559530798 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114040ba0 con 0x7fa1180ccbc0 2023-11-24T06:28:18.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.074+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa118157f70 con 0x7fa1180ccbc0 2023-11-24T06:28:18.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.075+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa118067b60 con 0x7fa1180ccbc0 2023-11-24T06:28:18.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.076+0000 7fa10effd640 1 -- 172.21.15.64:0/2559530798 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa114040d00 con 0x7fa1180ccbc0 2023-11-24T06:28:18.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.077+0000 7fa10effd640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa0fc06ae40 0x7fa0fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.077+0000 7fa10effd640 1 -- 172.21.15.64:0/2559530798 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1140ac460 con 0x7fa1180ccbc0 2023-11-24T06:28:18.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.077+0000 7fa11dc17640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa0fc06ae40 0x7fa0fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.078+0000 7fa11dc17640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa0fc06ae40 0x7fa0fc06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa11806e750 tx=0x7fa108009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.081+0000 7fa10effd640 1 -- 172.21.15.64:0/2559530798 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa114077620 con 0x7fa1180ccbc0 2023-11-24T06:28:18.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.280+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1180b2840 con 0x7fa1180ccbc0 2023-11-24T06:28:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.281+0000 7fa10effd640 1 -- 172.21.15.64:0/2559530798 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa114081070 con 0x7fa1180ccbc0 2023-11-24T06:28:18.284 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa0fc06ae40 msgr2=0x7fa0fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa0fc06ae40 0x7fa0fc06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa11806e750 tx=0x7fa108009040 comp rx=0 tx=0).stop 2023-11-24T06:28:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180ccbc0 msgr2=0x7fa118147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180ccbc0 0x7fa118147280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa114037040 tx=0x7fa114004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 shutdown_connections 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa0fc06ae40 0x7fa0fc06d300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa11815b320 0x7fa1180ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180ce5a0 0x7fa118158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 --2- 172.21.15.64:0/2559530798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180ccbc0 0x7fa118147280 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.284+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 >> 172.21.15.64:0/2559530798 conn(0x7fa1180b2a60 msgr2=0x7fa11806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.285+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 shutdown_connections 2023-11-24T06:28:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.285+0000 7fa11f41a640 1 -- 172.21.15.64:0/2559530798 wait complete. 2023-11-24T06:28:18.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:18.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.535+0000 7f8003e31640 1 Processor -- start 2023-11-24T06:28:18.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.535+0000 7f8003e31640 1 -- start start 2023-11-24T06:28:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8003e31640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ffc152250 0x7f7ffc15c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8003e31640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc06ebc0 0x7f7ffc06f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8003e31640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ffc0af150 con 0x7f7ffc06b3f0 2023-11-24T06:28:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8003e31640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ffc06f570 con 0x7f7ffc152250 2023-11-24T06:28:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8003e31640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0b2d80 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8001bcd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ffc152250 0x7f7ffc15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8001bcd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ffc152250 0x7f7ffc15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41652/0 (socket says 172.21.15.64:41652) 2023-11-24T06:28:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.537+0000 7f8001bcd640 1 -- 172.21.15.64:0/2185533554 learned_addr learned my addr 172.21.15.64:0/2185533554 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:18.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.538+0000 7f8001bcd640 1 -- 172.21.15.64:0/2185533554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc06ebc0 msgr2=0x7f7ffc06f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.538+0000 7f8001bcd640 1 --2- 172.21.15.64:0/2185533554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc06ebc0 0x7f7ffc06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.538+0000 7f8001bcd640 1 -- 172.21.15.64:0/2185533554 >> v1:172.21.15.64:6789/0 conn(0x7f7ffc06b3f0 legacy=0x7f7ffc0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.538+0000 7f8001bcd640 1 -- 172.21.15.64:0/2185533554 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc0ce050 con 0x7f7ffc152250 2023-11-24T06:28:18.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.539+0000 7f8001bcd640 1 --2- 172.21.15.64:0/2185533554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc152250 0x7f7ffc15c360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4970c37a6ea182d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.539+0000 7f8000bcb640 1 -- 172.21.15.64:0/2185533554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec031640 con 0x7f7ffc152250 2023-11-24T06:28:18.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.539+0000 7f8000bcb640 1 -- 172.21.15.64:0/2185533554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec0317a0 con 0x7f7ffc152250 2023-11-24T06:28:18.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.540+0000 7f8000bcb640 1 -- 172.21.15.64:0/2185533554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec031b30 con 0x7f7ffc152250 2023-11-24T06:28:18.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.540+0000 7f8003e31640 1 -- 172.21.15.64:0/2185533554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc152250 msgr2=0x7f7ffc15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.540+0000 7f8003e31640 1 --2- 172.21.15.64:0/2185533554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc152250 0x7f7ffc15c360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.540+0000 7f8003e31640 1 -- 172.21.15.64:0/2185533554 shutdown_connections 2023-11-24T06:28:18.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.540+0000 7f8003e31640 1 --2- 172.21.15.64:0/2185533554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc06ebc0 0x7f7ffc06f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.540+0000 7f8003e31640 1 --2- 172.21.15.64:0/2185533554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc152250 0x7f7ffc15c360 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.540+0000 7f8003e31640 1 -- 172.21.15.64:0/2185533554 >> 172.21.15.64:0/2185533554 conn(0x7f7ffc070b10 msgr2=0x7f7ffc070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.541+0000 7f8003e31640 1 -- 172.21.15.64:0/2185533554 shutdown_connections 2023-11-24T06:28:18.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.541+0000 7f8003e31640 1 -- 172.21.15.64:0/2185533554 wait complete. 2023-11-24T06:28:18.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.542+0000 7f8003e31640 1 Processor -- start 2023-11-24T06:28:18.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.543+0000 7f8003e31640 1 -- start start 2023-11-24T06:28:18.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.543+0000 7f8003e31640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ffc06b3f0 0x7f7ffc145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.543+0000 7f8003e31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc06ebc0 0x7f7ffc1f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f8003e31640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc152250 0x7f7ffc145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f8003e31640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc1f68b0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f8003e31640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc153870 con 0x7f7ffc06b3f0 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f8003e31640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc1f7280 con 0x7f7ffc152250 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc06ebc0 0x7f7ffc1f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc06ebc0 0x7f7ffc1f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41662/0 (socket says 172.21.15.64:41662) 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 -- 172.21.15.64:0/4137272217 learned_addr learned my addr 172.21.15.64:0/4137272217 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 -- 172.21.15.64:0/4137272217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc152250 msgr2=0x7f7ffc145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc152250 0x7f7ffc145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f8001bcd640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ffc06b3f0 0x7f7ffc145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 -- 172.21.15.64:0/4137272217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ffc06b3f0 msgr2=0x7f7ffc145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ffc06b3f0 0x7f7ffc145df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.544+0000 7f80013cc640 1 -- 172.21.15.64:0/4137272217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0094d0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.545+0000 7f80013cc640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc06ebc0 0x7f7ffc1f6d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000c400 tx=0x7f7ff000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.545+0000 7f7ffa7fc640 1 -- 172.21.15.64:0/4137272217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00052d0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.545+0000 7f7ffa7fc640 1 -- 172.21.15.64:0/4137272217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff0005430 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.545+0000 7f7ffa7fc640 1 -- 172.21.15.64:0/4137272217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0003830 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.545+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc1532b0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.545+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc1f6540 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.546+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ffc1f63a0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.547+0000 7f7ffa7fc640 1 -- 172.21.15.64:0/4137272217 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ff0003990 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.548+0000 7f7ffa7fc640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7fe806ae70 0x7f7fe806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.548+0000 7f7ffa7fc640 1 -- 172.21.15.64:0/4137272217 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ff0088b30 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.549+0000 7f8001bcd640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7fe806ae70 0x7f7fe806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.549+0000 7f8001bcd640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7fe806ae70 0x7f7fe806d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fec036000 tx=0x7f7fec03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.553+0000 7f7ffa7fc640 1 -- 172.21.15.64:0/4137272217 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ffc1f63a0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.747+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ffc1f63a0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.747+0000 7f7ffa7fc640 1 -- 172.21.15.64:0/4137272217 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ffc1f63a0 con 0x7f7ffc06ebc0 2023-11-24T06:28:18.750 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7fe806ae70 msgr2=0x7f7fe806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7fe806ae70 0x7f7fe806d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fec036000 tx=0x7f7fec03c040 comp rx=0 tx=0).stop 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc06ebc0 msgr2=0x7f7ffc1f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc06ebc0 0x7f7ffc1f6d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000c400 tx=0x7f7ff000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 shutdown_connections 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7fe806ae70 0x7f7fe806d330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ffc152250 0x7f7ffc145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc06ebc0 0x7f7ffc1f6d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 --2- 172.21.15.64:0/4137272217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ffc06b3f0 0x7f7ffc145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 >> 172.21.15.64:0/4137272217 conn(0x7f7ffc070b10 msgr2=0x7f7ffc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 shutdown_connections 2023-11-24T06:28:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:18.750+0000 7f8003e31640 1 -- 172.21.15.64:0/4137272217 wait complete. 2023-11-24T06:28:18.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:19.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.005+0000 7f49589ba640 1 Processor -- start 2023-11-24T06:28:19.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.006+0000 7f49589ba640 1 -- start start 2023-11-24T06:28:19.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.007+0000 7f49589ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f495415d4b0 0x7f49540d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.007+0000 7f49589ba640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cf9a0 0x7f49540cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.007+0000 7f49589ba640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f49540cddf0 0x7f49540ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.007+0000 7f49589ba640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49540d0320 con 0x7f495415d4b0 2023-11-24T06:28:19.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.007+0000 7f49589ba640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49540b2730 con 0x7f49540cddf0 2023-11-24T06:28:19.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.007+0000 7f49589ba640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49540d1ec0 con 0x7f49540cf9a0 2023-11-24T06:28:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.007+0000 7f49537fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f495415d4b0 0x7f49540d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f49537fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f495415d4b0 0x7f49540d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41678/0 (socket says 172.21.15.64:41678) 2023-11-24T06:28:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f49537fe640 1 -- 172.21.15.64:0/154625053 learned_addr learned my addr 172.21.15.64:0/154625053 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f49537fe640 1 -- 172.21.15.64:0/154625053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cf9a0 msgr2=0x7f49540cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f49537fe640 1 --2- 172.21.15.64:0/154625053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cf9a0 0x7f49540cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f49537fe640 1 -- 172.21.15.64:0/154625053 >> v2:172.21.15.162:3300/0 conn(0x7f49540cddf0 msgr2=0x7f49540ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f4953fff640 1 --2- 172.21.15.64:0/154625053 >> v2:172.21.15.162:3300/0 conn(0x7f49540cddf0 0x7f49540ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f49537fe640 1 --2- 172.21.15.64:0/154625053 >> v2:172.21.15.162:3300/0 conn(0x7f49540cddf0 0x7f49540ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.008+0000 7f49537fe640 1 -- 172.21.15.64:0/154625053 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540b29d0 con 0x7f495415d4b0 2023-11-24T06:28:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.009+0000 7f49537fe640 1 --2- 172.21.15.64:0/154625053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f495415d4b0 0x7f49540d1550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4944009e50 tx=0x7f494402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad5771abde1b16 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.009+0000 7f49527fc640 1 -- 172.21.15.64:0/154625053 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494402ca80 con 0x7f495415d4b0 2023-11-24T06:28:19.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49527fc640 1 -- 172.21.15.64:0/154625053 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494402cbe0 con 0x7f495415d4b0 2023-11-24T06:28:19.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49527fc640 1 -- 172.21.15.64:0/154625053 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440343f0 con 0x7f495415d4b0 2023-11-24T06:28:19.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 -- 172.21.15.64:0/154625053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f495415d4b0 msgr2=0x7f49540d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 --2- 172.21.15.64:0/154625053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f495415d4b0 0x7f49540d1550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4944009e50 tx=0x7f494402bee0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 -- 172.21.15.64:0/154625053 shutdown_connections 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 --2- 172.21.15.64:0/154625053 >> v2:172.21.15.162:3300/0 conn(0x7f49540cddf0 0x7f49540ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 --2- 172.21.15.64:0/154625053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cf9a0 0x7f49540cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 --2- 172.21.15.64:0/154625053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f495415d4b0 0x7f49540d1550 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 -- 172.21.15.64:0/154625053 >> 172.21.15.64:0/154625053 conn(0x7f49540d3000 msgr2=0x7f49540d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 -- 172.21.15.64:0/154625053 shutdown_connections 2023-11-24T06:28:19.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.010+0000 7f49589ba640 1 -- 172.21.15.64:0/154625053 wait complete. 2023-11-24T06:28:19.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.012+0000 7f49589ba640 1 Processor -- start 2023-11-24T06:28:19.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.012+0000 7f49589ba640 1 -- start start 2023-11-24T06:28:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.013+0000 7f49589ba640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cddf0 0x7f495409fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.013+0000 7f49589ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cf9a0 0x7f49540b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.013+0000 7f49589ba640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495415d4b0 0x7f49541553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.013+0000 7f49589ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954155f80 con 0x7f49540cf9a0 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.013+0000 7f49589ba640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954156260 con 0x7f495415d4b0 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.013+0000 7f49589ba640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49540a04d0 con 0x7f49540cddf0 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cf9a0 0x7f49540b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cf9a0 0x7f49540b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41694/0 (socket says 172.21.15.64:41694) 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 -- 172.21.15.64:0/458407186 learned_addr learned my addr 172.21.15.64:0/458407186 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f49537fe640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cddf0 0x7f495409fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4953fff640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495415d4b0 0x7f49541553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 -- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cddf0 msgr2=0x7f495409fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cddf0 0x7f495409fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 -- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495415d4b0 msgr2=0x7f49541553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495415d4b0 0x7f49541553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 -- 172.21.15.64:0/458407186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009b30 con 0x7f49540cf9a0 2023-11-24T06:28:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f49537fe640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cddf0 0x7f495409fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.014+0000 7f4952ffd640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cf9a0 0x7f49540b04c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f494800b4a0 tx=0x7f494800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.015+0000 7f493bfff640 1 -- 172.21.15.64:0/458407186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948010610 con 0x7f49540cf9a0 2023-11-24T06:28:19.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.015+0000 7f493bfff640 1 -- 172.21.15.64:0/458407186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948010770 con 0x7f49540cf9a0 2023-11-24T06:28:19.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.015+0000 7f493bfff640 1 -- 172.21.15.64:0/458407186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948010b20 con 0x7f49540cf9a0 2023-11-24T06:28:19.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.015+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495415da10 con 0x7f49540cf9a0 2023-11-24T06:28:19.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.015+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49540a0a90 con 0x7f49540cf9a0 2023-11-24T06:28:19.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.016+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49541563d0 con 0x7f49540cf9a0 2023-11-24T06:28:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.017+0000 7f493bfff640 1 -- 172.21.15.64:0/458407186 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4948010c80 con 0x7f49540cf9a0 2023-11-24T06:28:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.018+0000 7f493bfff640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f494006ae70 0x7f494006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.018+0000 7f493bfff640 1 -- 172.21.15.64:0/458407186 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49480891f0 con 0x7f49540cf9a0 2023-11-24T06:28:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.018+0000 7f49537fe640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f494006ae70 0x7f494006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.019+0000 7f49537fe640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f494006ae70 0x7f494006d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4944035000 tx=0x7f4944030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.022+0000 7f493bfff640 1 -- 172.21.15.64:0/458407186 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4948054290 con 0x7f49540cf9a0 2023-11-24T06:28:19.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.217+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4954067450 con 0x7f49540cf9a0 2023-11-24T06:28:19.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.218+0000 7f493bfff640 1 -- 172.21.15.64:0/458407186 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49480578b0 con 0x7f49540cf9a0 2023-11-24T06:28:19.221 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:19.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.220+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f494006ae70 msgr2=0x7f494006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.220+0000 7f49589ba640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f494006ae70 0x7f494006d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4944035000 tx=0x7f4944030000 comp rx=0 tx=0).stop 2023-11-24T06:28:19.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cf9a0 msgr2=0x7f49540b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cf9a0 0x7f49540b04c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f494800b4a0 tx=0x7f494800b960 comp rx=0 tx=0).stop 2023-11-24T06:28:19.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 shutdown_connections 2023-11-24T06:28:19.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f494006ae70 0x7f494006d330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495415d4b0 0x7f49541553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cf9a0 0x7f49540b04c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 --2- 172.21.15.64:0/458407186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49540cddf0 0x7f495409fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 >> 172.21.15.64:0/458407186 conn(0x7f49540d3000 msgr2=0x7f49540ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 shutdown_connections 2023-11-24T06:28:19.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.221+0000 7f49589ba640 1 -- 172.21.15.64:0/458407186 wait complete. 2023-11-24T06:28:19.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:19.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.467+0000 7f0a16494640 1 Processor -- start 2023-11-24T06:28:19.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.467+0000 7f0a16494640 1 -- start start 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.468+0000 7f0a16494640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a100cf9a0 0x7f0a1015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.468+0000 7f0a16494640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a1015e1e0 0x7f0a100702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.468+0000 7f0a16494640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a100b1ee0 con 0x7f0a10143ae0 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.468+0000 7f0a16494640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a100b2780 con 0x7f0a1015e1e0 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.469+0000 7f0a16494640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1015e990 con 0x7f0a100cf9a0 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.469+0000 7f0a14a31640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a1015e1e0 0x7f0a100702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.469+0000 7f0a14a31640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a1015e1e0 0x7f0a100702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:37450/0 (socket says 172.21.15.64:37450) 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.469+0000 7f0a14a31640 1 -- 172.21.15.64:0/110291577 learned_addr learned my addr 172.21.15.64:0/110291577 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.469+0000 7f0a077fe640 1 --2- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a100cf9a0 0x7f0a1015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.470+0000 7f0a14a31640 1 -- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a100cf9a0 msgr2=0x7f0a1015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:19.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.470+0000 7f0a14a31640 1 --2- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a100cf9a0 0x7f0a1015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.470+0000 7f0a14a31640 1 -- 172.21.15.64:0/110291577 >> v1:172.21.15.162:6789/0 conn(0x7f0a10143ae0 legacy=0x7f0a100cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.470+0000 7f0a14a31640 1 -- 172.21.15.64:0/110291577 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a100b2130 con 0x7f0a1015e1e0 2023-11-24T06:28:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.471+0000 7f0a14a31640 1 --2- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a1015e1e0 0x7f0a100702c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009870 tx=0x7f0a0802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ecfb809d33a0acd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.472+0000 7f0a0f7fe640 1 -- 172.21.15.64:0/110291577 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08030840 con 0x7f0a1015e1e0 2023-11-24T06:28:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.472+0000 7f0a0f7fe640 1 -- 172.21.15.64:0/110291577 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a080309a0 con 0x7f0a1015e1e0 2023-11-24T06:28:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.472+0000 7f0a0f7fe640 1 -- 172.21.15.64:0/110291577 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08030d30 con 0x7f0a1015e1e0 2023-11-24T06:28:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 -- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a1015e1e0 msgr2=0x7f0a100702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 --2- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a1015e1e0 0x7f0a100702c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009870 tx=0x7f0a0802b980 comp rx=0 tx=0).stop 2023-11-24T06:28:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 -- 172.21.15.64:0/110291577 shutdown_connections 2023-11-24T06:28:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 --2- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a100cf9a0 0x7f0a1015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 --2- 172.21.15.64:0/110291577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a1015e1e0 0x7f0a100702c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 -- 172.21.15.64:0/110291577 >> 172.21.15.64:0/110291577 conn(0x7f0a1006a800 msgr2=0x7f0a1006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 -- 172.21.15.64:0/110291577 shutdown_connections 2023-11-24T06:28:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.473+0000 7f0a16494640 1 -- 172.21.15.64:0/110291577 wait complete. 2023-11-24T06:28:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.475+0000 7f0a16494640 1 Processor -- start 2023-11-24T06:28:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.475+0000 7f0a16494640 1 -- start start 2023-11-24T06:28:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a16494640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a100cf9a0 0x7f0a10154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a16494640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a10143ae0 0x7f0a101599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a16494640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a1015e1e0 0x7f0a10139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a16494640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10135690 con 0x7f0a100cf9a0 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a16494640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1013eb40 con 0x7f0a10143ae0 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a16494640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1013c290 con 0x7f0a1015e1e0 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a100cf9a0 0x7f0a10154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a100cf9a0 0x7f0a10154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41706/0 (socket says 172.21.15.64:41706) 2023-11-24T06:28:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a0ffff640 1 -- 172.21.15.64:0/87727679 learned_addr learned my addr 172.21.15.64:0/87727679 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a14a31640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a1015e1e0 0x7f0a10139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.476+0000 7f0a077fe640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a10143ae0 0x7f0a101599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0ffff640 1 -- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a1015e1e0 msgr2=0x7f0a10139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0ffff640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a1015e1e0 0x7f0a10139180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0ffff640 1 -- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a10143ae0 msgr2=0x7f0a101599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0ffff640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a10143ae0 0x7f0a101599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0ffff640 1 -- 172.21.15.64:0/87727679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080094d0 con 0x7f0a100cf9a0 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0ffff640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a100cf9a0 0x7f0a10154fd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c870 tx=0x7f0a0000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0d7fa640 1 -- 172.21.15.64:0/87727679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00003b70 con 0x7f0a100cf9a0 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0d7fa640 1 -- 172.21.15.64:0/87727679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00003cf0 con 0x7f0a100cf9a0 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a0d7fa640 1 -- 172.21.15.64:0/87727679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0000aa90 con 0x7f0a100cf9a0 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a10155510 con 0x7f0a100cf9a0 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.477+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1013e860 con 0x7f0a100cf9a0 2023-11-24T06:28:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.479+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a100675d0 con 0x7f0a100cf9a0 2023-11-24T06:28:19.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.480+0000 7f0a0d7fa640 1 -- 172.21.15.64:0/87727679 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a0000abf0 con 0x7f0a100cf9a0 2023-11-24T06:28:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.481+0000 7f0a0d7fa640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f09f006ae40 0x7f09f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.481+0000 7f0a0d7fa640 1 -- 172.21.15.64:0/87727679 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a000894e0 con 0x7f0a100cf9a0 2023-11-24T06:28:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.481+0000 7f0a077fe640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f09f006ae40 0x7f09f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.482+0000 7f0a077fe640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f09f006ae40 0x7f09f006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0a1013ed50 tx=0x7f09f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.485+0000 7f0a0d7fa640 1 -- 172.21.15.64:0/87727679 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a00054580 con 0x7f0a100cf9a0 2023-11-24T06:28:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.683+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a100cebc0 con 0x7f0a100cf9a0 2023-11-24T06:28:19.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.686+0000 7f0a0d7fa640 1 -- 172.21.15.64:0/87727679 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f0a00003e60 con 0x7f0a100cf9a0 2023-11-24T06:28:19.690 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:19.690 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.689+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f09f006ae40 msgr2=0x7f09f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.689+0000 7f0a16494640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f09f006ae40 0x7f09f006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0a1013ed50 tx=0x7f09f8008040 comp rx=0 tx=0).stop 2023-11-24T06:28:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.689+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a100cf9a0 msgr2=0x7f0a10154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.689+0000 7f0a16494640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a100cf9a0 0x7f0a10154fd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c870 tx=0x7f0a0000cd30 comp rx=0 tx=0).stop 2023-11-24T06:28:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 shutdown_connections 2023-11-24T06:28:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f09f006ae40 0x7f09f006d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a1015e1e0 0x7f0a10139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a10143ae0 0x7f0a101599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 --2- 172.21.15.64:0/87727679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a100cf9a0 0x7f0a10154fd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 >> 172.21.15.64:0/87727679 conn(0x7f0a1006a800 msgr2=0x7f0a1006cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 shutdown_connections 2023-11-24T06:28:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.690+0000 7f0a16494640 1 -- 172.21.15.64:0/87727679 wait complete. 2023-11-24T06:28:19.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:19.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.929+0000 7fe109797640 1 Processor -- start 2023-11-24T06:28:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.930+0000 7fe109797640 1 -- start start 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.931+0000 7fe109797640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe10415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.931+0000 7fe109797640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe10413e360 0x7fe1040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.931+0000 7fe109797640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10406c470 con 0x7fe10415f9c0 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.931+0000 7fe109797640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10406c750 con 0x7fe10413e360 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.931+0000 7fe109797640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10406ca30 con 0x7fe10413df80 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.932+0000 7fe102ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe10415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.932+0000 7fe1027fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe10413e360 0x7fe1040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.932+0000 7fe102ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe10415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:43396/0 (socket says 172.21.15.64:43396) 2023-11-24T06:28:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.932+0000 7fe102ffd640 1 -- 172.21.15.64:0/1326719518 learned_addr learned my addr 172.21.15.64:0/1326719518 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.933+0000 7fe102ffd640 1 -- 172.21.15.64:0/1326719518 >> v2:172.21.15.162:3300/0 conn(0x7fe10413e360 msgr2=0x7fe1040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.933+0000 7fe102ffd640 1 --2- 172.21.15.64:0/1326719518 >> v2:172.21.15.162:3300/0 conn(0x7fe10413e360 0x7fe1040cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.933+0000 7fe102ffd640 1 -- 172.21.15.64:0/1326719518 >> v1:172.21.15.162:6789/0 conn(0x7fe10415f9c0 legacy=0x7fe1041408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.933+0000 7fe102ffd640 1 -- 172.21.15.64:0/1326719518 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10406cd10 con 0x7fe10413df80 2023-11-24T06:28:19.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.934+0000 7fe102ffd640 1 --2- 172.21.15.64:0/1326719518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe10415bee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009e50 tx=0x7fe0f402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92437574ccd3504 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.935+0000 7fe101ffb640 1 -- 172.21.15.64:0/1326719518 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f402cb00 con 0x7fe10413df80 2023-11-24T06:28:19.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.935+0000 7fe101ffb640 1 -- 172.21.15.64:0/1326719518 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f402cc60 con 0x7fe10413df80 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.935+0000 7fe101ffb640 1 -- 172.21.15.64:0/1326719518 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40343f0 con 0x7fe10413df80 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.935+0000 7fe109797640 1 -- 172.21.15.64:0/1326719518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 msgr2=0x7fe10415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.935+0000 7fe109797640 1 --2- 172.21.15.64:0/1326719518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe10415bee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009e50 tx=0x7fe0f402bf60 comp rx=0 tx=0).stop 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.936+0000 7fe109797640 1 -- 172.21.15.64:0/1326719518 shutdown_connections 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.936+0000 7fe109797640 1 --2- 172.21.15.64:0/1326719518 >> v2:172.21.15.162:3300/0 conn(0x7fe10413e360 0x7fe1040cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.936+0000 7fe109797640 1 --2- 172.21.15.64:0/1326719518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe10415bee0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.936+0000 7fe109797640 1 -- 172.21.15.64:0/1326719518 >> 172.21.15.64:0/1326719518 conn(0x7fe10406a800 msgr2=0x7fe10406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.936+0000 7fe109797640 1 -- 172.21.15.64:0/1326719518 shutdown_connections 2023-11-24T06:28:19.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.936+0000 7fe109797640 1 -- 172.21.15.64:0/1326719518 wait complete. 2023-11-24T06:28:19.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.938+0000 7fe109797640 1 Processor -- start 2023-11-24T06:28:19.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.938+0000 7fe109797640 1 -- start start 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe109797640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe1040ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe109797640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe10413e360 0x7fe10413d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe109797640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe10415f9c0 0x7fe10415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe109797640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1041580d0 con 0x7fe10415f9c0 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe109797640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe104153450 con 0x7fe10413e360 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe109797640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040702c0 con 0x7fe10413df80 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe1037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe10415f9c0 0x7fe10415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe1037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe10415f9c0 0x7fe10415dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41716/0 (socket says 172.21.15.64:41716) 2023-11-24T06:28:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe1037fe640 1 -- 172.21.15.64:0/1354545884 learned_addr learned my addr 172.21.15.64:0/1354545884 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe1027fc640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe10413e360 0x7fe10413d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.939+0000 7fe102ffd640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe1040ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe1037fe640 1 -- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 msgr2=0x7fe1040ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe1037fe640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe1040ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe1037fe640 1 -- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe10413e360 msgr2=0x7fe10413d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe1037fe640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe10413e360 0x7fe10413d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe1037fe640 1 -- 172.21.15.64:0/1354545884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4009b30 con 0x7fe10415f9c0 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe1037fe640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe10415f9c0 0x7fe10415dd70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c400 tx=0x7fe0fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe0fb7fe640 1 -- 172.21.15.64:0/1354545884 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0052d0 con 0x7fe10415f9c0 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe0fb7fe640 1 -- 172.21.15.64:0/1354545884 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc005430 con 0x7fe10415f9c0 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1041578c0 con 0x7fe10415f9c0 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.940+0000 7fe0fb7fe640 1 -- 172.21.15.64:0/1354545884 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc003810 con 0x7fe10415f9c0 2023-11-24T06:28:19.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.941+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe104157b10 con 0x7fe10415f9c0 2023-11-24T06:28:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.942+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10415ab70 con 0x7fe10415f9c0 2023-11-24T06:28:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.943+0000 7fe0fb7fe640 1 -- 172.21.15.64:0/1354545884 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0fc003970 con 0x7fe10415f9c0 2023-11-24T06:28:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.944+0000 7fe0fb7fe640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe0e406ae40 0x7fe0e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.944+0000 7fe0fb7fe640 1 -- 172.21.15.64:0/1354545884 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0fc088bc0 con 0x7fe10415f9c0 2023-11-24T06:28:19.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.944+0000 7fe102ffd640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe0e406ae40 0x7fe0e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.945+0000 7fe102ffd640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe0e406ae40 0x7fe0e406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4035000 tx=0x7fe0f4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:19.949+0000 7fe0fb7fe640 1 -- 172.21.15.64:0/1354545884 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe10415ab70 con 0x7fe10415f9c0 2023-11-24T06:28:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.147+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe10415ab70 con 0x7fe10415f9c0 2023-11-24T06:28:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.148+0000 7fe0fb7fe640 1 -- 172.21.15.64:0/1354545884 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe10415ab70 con 0x7fe10415f9c0 2023-11-24T06:28:20.151 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.150+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe0e406ae40 msgr2=0x7fe0e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.150+0000 7fe109797640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe0e406ae40 0x7fe0e406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4035000 tx=0x7fe0f4030000 comp rx=0 tx=0).stop 2023-11-24T06:28:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.150+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe10415f9c0 msgr2=0x7fe10415dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.150+0000 7fe109797640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe10415f9c0 0x7fe10415dd70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c400 tx=0x7fe0fc00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 shutdown_connections 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe0e406ae40 0x7fe0e406d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe10415f9c0 0x7fe10415dd70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe10413e360 0x7fe10413d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 --2- 172.21.15.64:0/1354545884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10413df80 0x7fe1040ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 >> 172.21.15.64:0/1354545884 conn(0x7fe10406a800 msgr2=0x7fe10415c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 shutdown_connections 2023-11-24T06:28:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.151+0000 7fe109797640 1 -- 172.21.15.64:0/1354545884 wait complete. 2023-11-24T06:28:20.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:20.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.403+0000 7ff4d58ea640 1 Processor -- start 2023-11-24T06:28:20.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.403+0000 7ff4d58ea640 1 -- start start 2023-11-24T06:28:20.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d58ea640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff4d00ccb50 0x7ff4d0070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d58ea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4d015bf10 0x7ff4d0143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d58ea640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d015f4e0 con 0x7ff4d00adf20 2023-11-24T06:28:20.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d58ea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d015f970 con 0x7ff4d015bf10 2023-11-24T06:28:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d58ea640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d0071130 con 0x7ff4d00ccb50 2023-11-24T06:28:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d50e9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4d015bf10 0x7ff4d0143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d50e9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4d015bf10 0x7ff4d0143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41720/0 (socket says 172.21.15.64:41720) 2023-11-24T06:28:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4d50e9640 1 -- 172.21.15.64:0/721314670 learned_addr learned my addr 172.21.15.64:0/721314670 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.406+0000 7ff4cffff640 1 --2- 172.21.15.64:0/721314670 >> v2:172.21.15.162:3300/0 conn(0x7ff4d00ccb50 0x7ff4d0070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.407+0000 7ff4d50e9640 1 -- 172.21.15.64:0/721314670 >> v2:172.21.15.162:3300/0 conn(0x7ff4d00ccb50 msgr2=0x7ff4d0070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.407+0000 7ff4d50e9640 1 --2- 172.21.15.64:0/721314670 >> v2:172.21.15.162:3300/0 conn(0x7ff4d00ccb50 0x7ff4d0070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.407+0000 7ff4d50e9640 1 -- 172.21.15.64:0/721314670 >> v1:172.21.15.162:6789/0 conn(0x7ff4d00adf20 legacy=0x7ff4d00cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.407+0000 7ff4d50e9640 1 -- 172.21.15.64:0/721314670 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d015ebb0 con 0x7ff4d015bf10 2023-11-24T06:28:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.408+0000 7ff4d50e9640 1 --2- 172.21.15.64:0/721314670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d015bf10 0x7ff4d0143d30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4d015e9e0 tx=0x7ff4c802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb1a5021ccd7e87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.408+0000 7ff4cf7fe640 1 -- 172.21.15.64:0/721314670 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c802c9f0 con 0x7ff4d015bf10 2023-11-24T06:28:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.408+0000 7ff4cf7fe640 1 -- 172.21.15.64:0/721314670 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c802cb50 con 0x7ff4d015bf10 2023-11-24T06:28:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.408+0000 7ff4cf7fe640 1 -- 172.21.15.64:0/721314670 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80343f0 con 0x7ff4d015bf10 2023-11-24T06:28:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 -- 172.21.15.64:0/721314670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d015bf10 msgr2=0x7ff4d0143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/721314670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d015bf10 0x7ff4d0143d30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4d015e9e0 tx=0x7ff4c802be50 comp rx=0 tx=0).stop 2023-11-24T06:28:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 -- 172.21.15.64:0/721314670 shutdown_connections 2023-11-24T06:28:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/721314670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d015bf10 0x7ff4d0143d30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/721314670 >> v2:172.21.15.162:3300/0 conn(0x7ff4d00ccb50 0x7ff4d0070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 -- 172.21.15.64:0/721314670 >> 172.21.15.64:0/721314670 conn(0x7ff4d006a7d0 msgr2=0x7ff4d006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 -- 172.21.15.64:0/721314670 shutdown_connections 2023-11-24T06:28:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.409+0000 7ff4d58ea640 1 -- 172.21.15.64:0/721314670 wait complete. 2023-11-24T06:28:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.411+0000 7ff4d58ea640 1 Processor -- start 2023-11-24T06:28:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.411+0000 7ff4d58ea640 1 -- start start 2023-11-24T06:28:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.412+0000 7ff4d58ea640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4d00adf20 0x7ff4d0140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.412+0000 7ff4d58ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d00ccb50 0x7ff4d0132390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.412+0000 7ff4d58ea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4d015bf10 0x7ff4d0134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.412+0000 7ff4d58ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d015af90 con 0x7ff4d00ccb50 2023-11-24T06:28:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.412+0000 7ff4d58ea640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0133640 con 0x7ff4d00adf20 2023-11-24T06:28:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.412+0000 7ff4d58ea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0135390 con 0x7ff4d015bf10 2023-11-24T06:28:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d00ccb50 0x7ff4d0132390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d00ccb50 0x7ff4d0132390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41736/0 (socket says 172.21.15.64:41736) 2023-11-24T06:28:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 -- 172.21.15.64:0/2401194719 learned_addr learned my addr 172.21.15.64:0/2401194719 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4d48e8640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4d00adf20 0x7ff4d0140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4d50e9640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4d015bf10 0x7ff4d0134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 -- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4d015bf10 msgr2=0x7ff4d0134e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4d015bf10 0x7ff4d0134e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 -- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4d00adf20 msgr2=0x7ff4d0140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4d00adf20 0x7ff4d0140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 -- 172.21.15.64:0/2401194719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c8009b30 con 0x7ff4d00ccb50 2023-11-24T06:28:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.413+0000 7ff4cffff640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d00ccb50 0x7ff4d0132390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d7d0 tx=0x7ff4bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.414+0000 7ff4cd7fa640 1 -- 172.21.15.64:0/2401194719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00b6c0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.414+0000 7ff4cd7fa640 1 -- 172.21.15.64:0/2401194719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc00b820 con 0x7ff4d00ccb50 2023-11-24T06:28:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.414+0000 7ff4cd7fa640 1 -- 172.21.15.64:0/2401194719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00bbb0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.414+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d0132a30 con 0x7ff4d00ccb50 2023-11-24T06:28:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.414+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d0132ff0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.415+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d01337f0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.416+0000 7ff4cd7fa640 1 -- 172.21.15.64:0/2401194719 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4bc00bd10 con 0x7ff4d00ccb50 2023-11-24T06:28:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.417+0000 7ff4cd7fa640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff4b406ae70 0x7ff4b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.417+0000 7ff4cd7fa640 1 -- 172.21.15.64:0/2401194719 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4bc089bd0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.417+0000 7ff4d48e8640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff4b406ae70 0x7ff4b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.418+0000 7ff4d48e8640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff4b406ae70 0x7ff4b406d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004ea0 tx=0x7ff4c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.421+0000 7ff4cd7fa640 1 -- 172.21.15.64:0/2401194719 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4d01337f0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.616+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4d01337b0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.617+0000 7ff4cd7fa640 1 -- 172.21.15.64:0/2401194719 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff4bc054be0 con 0x7ff4d00ccb50 2023-11-24T06:28:20.620 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:20.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff4b406ae70 msgr2=0x7ff4b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff4b406ae70 0x7ff4b406d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004ea0 tx=0x7ff4c4009040 comp rx=0 tx=0).stop 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d00ccb50 msgr2=0x7ff4d0132390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d00ccb50 0x7ff4d0132390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d7d0 tx=0x7ff4bc00dc90 comp rx=0 tx=0).stop 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 shutdown_connections 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff4b406ae70 0x7ff4b406d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4d015bf10 0x7ff4d0134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4d00ccb50 0x7ff4d0132390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 --2- 172.21.15.64:0/2401194719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4d00adf20 0x7ff4d0140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 >> 172.21.15.64:0/2401194719 conn(0x7ff4d006a7d0 msgr2=0x7ff4d006c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.620+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 shutdown_connections 2023-11-24T06:28:20.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.621+0000 7ff4d58ea640 1 -- 172.21.15.64:0/2401194719 wait complete. 2023-11-24T06:28:20.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.878+0000 7f7b5a9cf640 1 Processor -- start 2023-11-24T06:28:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.878+0000 7f7b5a9cf640 1 -- start start 2023-11-24T06:28:20.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.879+0000 7f7b5a9cf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b5415d4b0 0x7f7b540d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.879+0000 7f7b5a9cf640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b540b41e0 0x7f7b540cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.880+0000 7f7b5a9cf640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b540d0320 con 0x7f7b540cf9a0 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.880+0000 7f7b5a9cf640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540b2730 con 0x7f7b540b41e0 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.880+0000 7f7b5a9cf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d1ec0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.880+0000 7f7b599cd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b5415d4b0 0x7f7b540d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.880+0000 7f7b5a1ce640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b540b41e0 0x7f7b540cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.880+0000 7f7b599cd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b5415d4b0 0x7f7b540d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:43426/0 (socket says 172.21.15.64:43426) 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.880+0000 7f7b599cd640 1 -- 172.21.15.64:0/1642746023 learned_addr learned my addr 172.21.15.64:0/1642746023 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.881+0000 7f7b599cd640 1 -- 172.21.15.64:0/1642746023 >> v2:172.21.15.162:3300/0 conn(0x7f7b540b41e0 msgr2=0x7f7b540cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.881+0000 7f7b599cd640 1 --2- 172.21.15.64:0/1642746023 >> v2:172.21.15.162:3300/0 conn(0x7f7b540b41e0 0x7f7b540cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.881+0000 7f7b599cd640 1 -- 172.21.15.64:0/1642746023 >> v1:172.21.15.162:6789/0 conn(0x7f7b540cf9a0 legacy=0x7f7b540cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.881+0000 7f7b599cd640 1 -- 172.21.15.64:0/1642746023 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540b29d0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.882+0000 7f7b599cd640 1 --2- 172.21.15.64:0/1642746023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b5415d4b0 0x7f7b540d1550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b500098a0 tx=0x7f7b5002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7dd1837a9512683 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b589cb640 1 -- 172.21.15.64:0/1642746023 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50030840 con 0x7f7b5415d4b0 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b589cb640 1 -- 172.21.15.64:0/1642746023 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b500309a0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b589cb640 1 -- 172.21.15.64:0/1642746023 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50030d30 con 0x7f7b5415d4b0 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/1642746023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b5415d4b0 msgr2=0x7f7b540d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/1642746023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b5415d4b0 0x7f7b540d1550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b500098a0 tx=0x7f7b5002b980 comp rx=0 tx=0).stop 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/1642746023 shutdown_connections 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/1642746023 >> v2:172.21.15.162:3300/0 conn(0x7f7b540b41e0 0x7f7b540cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/1642746023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b5415d4b0 0x7f7b540d1550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.883+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/1642746023 >> 172.21.15.64:0/1642746023 conn(0x7f7b540d3000 msgr2=0x7f7b540d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:20.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.884+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/1642746023 shutdown_connections 2023-11-24T06:28:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.884+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/1642746023 wait complete. 2023-11-24T06:28:20.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.886+0000 7f7b5a9cf640 1 Processor -- start 2023-11-24T06:28:20.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.886+0000 7f7b5a9cf640 1 -- start start 2023-11-24T06:28:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a9cf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b540b41e0 0x7f7b5414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a9cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b540cf9a0 0x7f7b5415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a9cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b5415d4b0 0x7f7b5414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a9cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b54160e90 con 0x7f7b5415d4b0 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a9cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5415fb50 con 0x7f7b540cf9a0 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a9cf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5414b2e0 con 0x7f7b540b41e0 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a1ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b5415d4b0 0x7f7b5414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a1ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b5415d4b0 0x7f7b5414ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41738/0 (socket says 172.21.15.64:41738) 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a1ce640 1 -- 172.21.15.64:0/939462341 learned_addr learned my addr 172.21.15.64:0/939462341 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b599cd640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b540b41e0 0x7f7b5414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b591cc640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b540cf9a0 0x7f7b5415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a1ce640 1 -- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b540b41e0 msgr2=0x7f7b5414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.888+0000 7f7b5a1ce640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b540b41e0 0x7f7b5414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b5a1ce640 1 -- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b540cf9a0 msgr2=0x7f7b5415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b5a1ce640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b540cf9a0 0x7f7b5415f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b5a1ce640 1 -- 172.21.15.64:0/939462341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500094d0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b5a1ce640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b5415d4b0 0x7f7b5414ada0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00efc0 tx=0x7f7b4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b4a7fc640 1 -- 172.21.15.64:0/939462341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c009ca0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b4a7fc640 1 -- 172.21.15.64:0/939462341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00f040 con 0x7f7b5415d4b0 2023-11-24T06:28:20.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b4a7fc640 1 -- 172.21.15.64:0/939462341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0146c0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5415da10 con 0x7f7b5415d4b0 2023-11-24T06:28:20.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.889+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b541600d0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.891+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5404ed60 con 0x7f7b5415d4b0 2023-11-24T06:28:20.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.892+0000 7f7b4a7fc640 1 -- 172.21.15.64:0/939462341 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b4c014820 con 0x7f7b5415d4b0 2023-11-24T06:28:20.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.892+0000 7f7b4a7fc640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b3806ae40 0x7f7b3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.893+0000 7f7b4a7fc640 1 -- 172.21.15.64:0/939462341 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b4c08a5b0 con 0x7f7b5415d4b0 2023-11-24T06:28:20.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.893+0000 7f7b599cd640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b3806ae40 0x7f7b3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.893+0000 7f7b599cd640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b3806ae40 0x7f7b3806d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b50036000 tx=0x7f7b5003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:20.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:20.897+0000 7f7b4a7fc640 1 -- 172.21.15.64:0/939462341 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b4c055660 con 0x7f7b5415d4b0 2023-11-24T06:28:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.091+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b54067450 con 0x7f7b5415d4b0 2023-11-24T06:28:21.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.092+0000 7f7b4a7fc640 1 -- 172.21.15.64:0/939462341 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b4c058c80 con 0x7f7b5415d4b0 2023-11-24T06:28:21.095 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:21.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b3806ae40 msgr2=0x7f7b3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b3806ae40 0x7f7b3806d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b50036000 tx=0x7f7b5003b040 comp rx=0 tx=0).stop 2023-11-24T06:28:21.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b5415d4b0 msgr2=0x7f7b5414ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b5415d4b0 0x7f7b5414ada0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00efc0 tx=0x7f7b4c00c490 comp rx=0 tx=0).stop 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 shutdown_connections 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b3806ae40 0x7f7b3806d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b5415d4b0 0x7f7b5414ada0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b540cf9a0 0x7f7b5415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 --2- 172.21.15.64:0/939462341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b540b41e0 0x7f7b5414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 >> 172.21.15.64:0/939462341 conn(0x7f7b540d3000 msgr2=0x7f7b5406c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 shutdown_connections 2023-11-24T06:28:21.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.095+0000 7f7b5a9cf640 1 -- 172.21.15.64:0/939462341 wait complete. 2023-11-24T06:28:21.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:21.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.344+0000 7fa75083c640 1 Processor -- start 2023-11-24T06:28:21.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.344+0000 7fa75083c640 1 -- start start 2023-11-24T06:28:21.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.345+0000 7fa75083c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa74c143c40 0x7fa74c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa75083c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0717d0 0x7fa74c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa75083c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa74c0cca40 0x7fa74c15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa75083c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74c0723c0 con 0x7fa74c0cca40 2023-11-24T06:28:21.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa75083c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74c070e50 con 0x7fa74c143c40 2023-11-24T06:28:21.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa75083c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0cce70 con 0x7fa74c0717d0 2023-11-24T06:28:21.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa74c0cca40 0x7fa74c15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa74c0cca40 0x7fa74c15eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41744/0 (socket says 172.21.15.64:41744) 2023-11-24T06:28:21.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 -- 172.21.15.64:0/1829105001 learned_addr learned my addr 172.21.15.64:0/1829105001 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 -- 172.21.15.64:0/1829105001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0717d0 msgr2=0x7fa74c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa743fff640 1 --2- 172.21.15.64:0/1829105001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0717d0 0x7fa74c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74ad9d640 1 --2- 172.21.15.64:0/1829105001 >> v2:172.21.15.162:3300/0 conn(0x7fa74c143c40 0x7fa74c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 --2- 172.21.15.64:0/1829105001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0717d0 0x7fa74c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 -- 172.21.15.64:0/1829105001 >> v2:172.21.15.162:3300/0 conn(0x7fa74c143c40 msgr2=0x7fa74c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 --2- 172.21.15.64:0/1829105001 >> v2:172.21.15.162:3300/0 conn(0x7fa74c143c40 0x7fa74c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa74b59e640 1 -- 172.21.15.64:0/1829105001 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c144020 con 0x7fa74c0cca40 2023-11-24T06:28:21.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.346+0000 7fa743fff640 1 --2- 172.21.15.64:0/1829105001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0717d0 0x7fa74c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:21.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.348+0000 7fa74b59e640 1 --2- 172.21.15.64:0/1829105001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c0cca40 0x7fa74c15eab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa74c06e410 tx=0x7fa74402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764907dc10239b2b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.348+0000 7fa74a59c640 1 -- 172.21.15.64:0/1829105001 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74402c9e0 con 0x7fa74c0cca40 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.348+0000 7fa74a59c640 1 -- 172.21.15.64:0/1829105001 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74402cb40 con 0x7fa74c0cca40 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.348+0000 7fa74a59c640 1 -- 172.21.15.64:0/1829105001 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7440343f0 con 0x7fa74c0cca40 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.348+0000 7fa75083c640 1 -- 172.21.15.64:0/1829105001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c0cca40 msgr2=0x7fa74c15eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.348+0000 7fa75083c640 1 --2- 172.21.15.64:0/1829105001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c0cca40 0x7fa74c15eab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa74c06e410 tx=0x7fa74402be40 comp rx=0 tx=0).stop 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.349+0000 7fa75083c640 1 -- 172.21.15.64:0/1829105001 shutdown_connections 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.349+0000 7fa75083c640 1 --2- 172.21.15.64:0/1829105001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c0cca40 0x7fa74c15eab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.349+0000 7fa75083c640 1 --2- 172.21.15.64:0/1829105001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0717d0 0x7fa74c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.349+0000 7fa75083c640 1 --2- 172.21.15.64:0/1829105001 >> v2:172.21.15.162:3300/0 conn(0x7fa74c143c40 0x7fa74c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.349+0000 7fa75083c640 1 -- 172.21.15.64:0/1829105001 >> 172.21.15.64:0/1829105001 conn(0x7fa74c06a800 msgr2=0x7fa74c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.349+0000 7fa75083c640 1 -- 172.21.15.64:0/1829105001 shutdown_connections 2023-11-24T06:28:21.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.349+0000 7fa75083c640 1 -- 172.21.15.64:0/1829105001 wait complete. 2023-11-24T06:28:21.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.350+0000 7fa75083c640 1 Processor -- start 2023-11-24T06:28:21.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.351+0000 7fa75083c640 1 -- start start 2023-11-24T06:28:21.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa75083c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa74c0717d0 0x7fa74c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa75083c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0cca40 0x7fa74c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa75083c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c143c40 0x7fa74c13aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa75083c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0b3fd0 con 0x7fa74c143c40 2023-11-24T06:28:21.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa75083c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c138430 con 0x7fa74c0717d0 2023-11-24T06:28:21.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa75083c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0b3a10 con 0x7fa74c0cca40 2023-11-24T06:28:21.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c143c40 0x7fa74c13aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c143c40 0x7fa74c13aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41746/0 (socket says 172.21.15.64:41746) 2023-11-24T06:28:21.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 -- 172.21.15.64:0/687010780 learned_addr learned my addr 172.21.15.64:0/687010780 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 -- 172.21.15.64:0/687010780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0cca40 msgr2=0x7fa74c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0cca40 0x7fa74c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 -- 172.21.15.64:0/687010780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa74c0717d0 msgr2=0x7fa74c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa74c0717d0 0x7fa74c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.352+0000 7fa74b59e640 1 -- 172.21.15.64:0/687010780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744009b30 con 0x7fa74c143c40 2023-11-24T06:28:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.353+0000 7fa74b59e640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c143c40 0x7fa74c13aa30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa744036040 tx=0x7fa744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.353+0000 7fa7437fe640 1 -- 172.21.15.64:0/687010780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744009510 con 0x7fa74c143c40 2023-11-24T06:28:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.353+0000 7fa7437fe640 1 -- 172.21.15.64:0/687010780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74403e450 con 0x7fa74c143c40 2023-11-24T06:28:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.353+0000 7fa7437fe640 1 -- 172.21.15.64:0/687010780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74403e800 con 0x7fa74c143c40 2023-11-24T06:28:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.353+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c0b3cf0 con 0x7fa74c143c40 2023-11-24T06:28:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.353+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c15a550 con 0x7fa74c143c40 2023-11-24T06:28:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.354+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c067db0 con 0x7fa74c143c40 2023-11-24T06:28:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.355+0000 7fa7437fe640 1 -- 172.21.15.64:0/687010780 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa74403e960 con 0x7fa74c143c40 2023-11-24T06:28:21.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.356+0000 7fa7437fe640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa72c06ae40 0x7fa72c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.356+0000 7fa7437fe640 1 -- 172.21.15.64:0/687010780 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7440ac5b0 con 0x7fa74c143c40 2023-11-24T06:28:21.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.356+0000 7fa74ad9d640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa72c06ae40 0x7fa72c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.357+0000 7fa74ad9d640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa72c06ae40 0x7fa72c06d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa73c006b10 tx=0x7fa73c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.360+0000 7fa7437fe640 1 -- 172.21.15.64:0/687010780 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa744077670 con 0x7fa74c143c40 2023-11-24T06:28:21.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.559+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa74c13a370 con 0x7fa74c143c40 2023-11-24T06:28:21.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.561+0000 7fa7437fe640 1 -- 172.21.15.64:0/687010780 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fa74c13a370 con 0x7fa74c143c40 2023-11-24T06:28:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:21.565 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:21.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa72c06ae40 msgr2=0x7fa72c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa72c06ae40 0x7fa72c06d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa73c006b10 tx=0x7fa73c008040 comp rx=0 tx=0).stop 2023-11-24T06:28:21.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c143c40 msgr2=0x7fa74c13aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c143c40 0x7fa74c13aa30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa744036040 tx=0x7fa744004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 shutdown_connections 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa72c06ae40 0x7fa72c06d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa74c143c40 0x7fa74c13aa30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa74c0cca40 0x7fa74c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 --2- 172.21.15.64:0/687010780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa74c0717d0 0x7fa74c15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.564+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 >> 172.21.15.64:0/687010780 conn(0x7fa74c06a800 msgr2=0x7fa74c144210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.565+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 shutdown_connections 2023-11-24T06:28:21.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.565+0000 7fa75083c640 1 -- 172.21.15.64:0/687010780 wait complete. 2023-11-24T06:28:21.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:21.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.816+0000 7f4eaae1a640 1 Processor -- start 2023-11-24T06:28:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.816+0000 7f4eaae1a640 1 -- start start 2023-11-24T06:28:21.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.818+0000 7f4eaae1a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ea40ccb50 0x7f4ea4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.818+0000 7f4eaae1a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.818+0000 7f4eaae1a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea415f4e0 con 0x7f4ea40adf20 2023-11-24T06:28:21.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.818+0000 7f4eaae1a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea415f970 con 0x7f4ea40ccb50 2023-11-24T06:28:21.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.818+0000 7f4eaae1a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415c3c0 con 0x7f4ea415bf10 2023-11-24T06:28:21.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.819+0000 7f4eaa619640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.819+0000 7f4eaa619640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea4143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:43450/0 (socket says 172.21.15.64:43450) 2023-11-24T06:28:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.819+0000 7f4eaa619640 1 -- 172.21.15.64:0/1742016036 learned_addr learned my addr 172.21.15.64:0/1742016036 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.819+0000 7f4ea9617640 1 --2- 172.21.15.64:0/1742016036 >> v2:172.21.15.162:3300/0 conn(0x7f4ea40ccb50 0x7f4ea4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.819+0000 7f4eaa619640 1 -- 172.21.15.64:0/1742016036 >> v2:172.21.15.162:3300/0 conn(0x7f4ea40ccb50 msgr2=0x7f4ea4070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.819+0000 7f4eaa619640 1 --2- 172.21.15.64:0/1742016036 >> v2:172.21.15.162:3300/0 conn(0x7f4ea40ccb50 0x7f4ea4070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.819+0000 7f4eaa619640 1 -- 172.21.15.64:0/1742016036 >> v1:172.21.15.162:6789/0 conn(0x7f4ea40adf20 legacy=0x7f4ea40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.820+0000 7f4eaa619640 1 -- 172.21.15.64:0/1742016036 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4071130 con 0x7f4ea415bf10 2023-11-24T06:28:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.821+0000 7f4eaa619640 1 --2- 172.21.15.64:0/1742016036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea4143d30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009e50 tx=0x7f4e9c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d82b2bc493f3472 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.821+0000 7f4ea8e16640 1 -- 172.21.15.64:0/1742016036 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c02ca70 con 0x7f4ea415bf10 2023-11-24T06:28:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.821+0000 7f4ea8e16640 1 -- 172.21.15.64:0/1742016036 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c02cbd0 con 0x7f4ea415bf10 2023-11-24T06:28:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.821+0000 7f4ea8e16640 1 -- 172.21.15.64:0/1742016036 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0343f0 con 0x7f4ea415bf10 2023-11-24T06:28:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 -- 172.21.15.64:0/1742016036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 msgr2=0x7f4ea4143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/1742016036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea4143d30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009e50 tx=0x7f4e9c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 -- 172.21.15.64:0/1742016036 shutdown_connections 2023-11-24T06:28:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/1742016036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea4143d30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/1742016036 >> v2:172.21.15.162:3300/0 conn(0x7f4ea40ccb50 0x7f4ea4070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 -- 172.21.15.64:0/1742016036 >> 172.21.15.64:0/1742016036 conn(0x7f4ea406a7d0 msgr2=0x7f4ea406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 -- 172.21.15.64:0/1742016036 shutdown_connections 2023-11-24T06:28:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.822+0000 7f4eaae1a640 1 -- 172.21.15.64:0/1742016036 wait complete. 2023-11-24T06:28:21.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.824+0000 7f4eaae1a640 1 Processor -- start 2023-11-24T06:28:21.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.825+0000 7f4eaae1a640 1 -- start start 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4eaae1a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea40adf20 0x7f4ea415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4eaae1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ea40ccb50 0x7f4ea40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4eaae1a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea41f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4eaae1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40b24a0 con 0x7f4ea40ccb50 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4eaae1a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40b1e90 con 0x7f4ea40adf20 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4eaae1a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea41f3410 con 0x7f4ea415bf10 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4ea9617640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ea40ccb50 0x7f4ea40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4ea9617640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ea40ccb50 0x7f4ea40b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41754/0 (socket says 172.21.15.64:41754) 2023-11-24T06:28:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.826+0000 7f4ea9617640 1 -- 172.21.15.64:0/4213112636 learned_addr learned my addr 172.21.15.64:0/4213112636 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4eaa619640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea41f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4ea9e18640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea40adf20 0x7f4ea415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4ea9617640 1 -- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 msgr2=0x7f4ea41f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4ea9617640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea41f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4ea9617640 1 -- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea40adf20 msgr2=0x7f4ea415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4ea9617640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea40adf20 0x7f4ea415a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4ea9617640 1 -- 172.21.15.64:0/4213112636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c009b30 con 0x7f4ea40ccb50 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.827+0000 7f4ea9617640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ea40ccb50 0x7f4ea40b28c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400e980 tx=0x7f4e9400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.828+0000 7f4e9a7fc640 1 -- 172.21.15.64:0/4213112636 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94009670 con 0x7f4ea40ccb50 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.828+0000 7f4e9a7fc640 1 -- 172.21.15.64:0/4213112636 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400ca00 con 0x7f4ea40ccb50 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.828+0000 7f4e9a7fc640 1 -- 172.21.15.64:0/4213112636 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400cd90 con 0x7f4ea40ccb50 2023-11-24T06:28:21.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.828+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea41f36f0 con 0x7f4ea40ccb50 2023-11-24T06:28:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.828+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea40b3c80 con 0x7f4ea40ccb50 2023-11-24T06:28:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.829+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea4067450 con 0x7f4ea40ccb50 2023-11-24T06:28:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.830+0000 7f4e9a7fc640 1 -- 172.21.15.64:0/4213112636 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e94013470 con 0x7f4ea40ccb50 2023-11-24T06:28:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.831+0000 7f4e9a7fc640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4e8806ae40 0x7f4e8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.831+0000 7f4e9a7fc640 1 -- 172.21.15.64:0/4213112636 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e940898f0 con 0x7f4ea40ccb50 2023-11-24T06:28:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.831+0000 7f4ea9e18640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4e8806ae40 0x7f4e8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.832+0000 7f4ea9e18640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4e8806ae40 0x7f4e8806d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000f60 tx=0x7f4ea0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:21.835+0000 7f4e9a7fc640 1 -- 172.21.15.64:0/4213112636 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e94054990 con 0x7f4ea40ccb50 2023-11-24T06:28:22.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.033+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ea41f38e0 con 0x7f4ea40ccb50 2023-11-24T06:28:22.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.034+0000 7f4e9a7fc640 1 -- 172.21.15.64:0/4213112636 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ea41f38e0 con 0x7f4ea40ccb50 2023-11-24T06:28:22.037 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4e8806ae40 msgr2=0x7f4e8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4e8806ae40 0x7f4e8806d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000f60 tx=0x7f4ea0009040 comp rx=0 tx=0).stop 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ea40ccb50 msgr2=0x7f4ea40b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ea40ccb50 0x7f4ea40b28c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400e980 tx=0x7f4e9400ee40 comp rx=0 tx=0).stop 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 shutdown_connections 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4e8806ae40 0x7f4e8806d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea415bf10 0x7f4ea41f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ea40ccb50 0x7f4ea40b28c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 --2- 172.21.15.64:0/4213112636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea40adf20 0x7f4ea415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 >> 172.21.15.64:0/4213112636 conn(0x7f4ea406a7d0 msgr2=0x7f4ea415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:22.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 shutdown_connections 2023-11-24T06:28:22.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.037+0000 7f4eaae1a640 1 -- 172.21.15.64:0/4213112636 wait complete. 2023-11-24T06:28:22.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.287+0000 7f0404a31640 1 Processor -- start 2023-11-24T06:28:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.287+0000 7f0404a31640 1 -- start start 2023-11-24T06:28:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.289+0000 7f0404a31640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0400072230 0x7f0400150760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.289+0000 7f0404a31640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04000cf290 0x7f040015d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.289+0000 7f0404a31640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0400071190 con 0x7f04000cf6d0 2023-11-24T06:28:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.289+0000 7f0404a31640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0400072610 con 0x7f0400072230 2023-11-24T06:28:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.289+0000 7f0404a31640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0400071470 con 0x7f04000cf290 2023-11-24T06:28:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.290+0000 7f03fed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f04000cf6d0 0x7f04000b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58222/0 (socket says 172.21.15.64:58222) 2023-11-24T06:28:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.290+0000 7f03fed9d640 1 -- 172.21.15.64:0/1802122058 learned_addr learned my addr 172.21.15.64:0/1802122058 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.290+0000 7f03fdd9b640 1 --2- 172.21.15.64:0/1802122058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04000cf290 0x7f040015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.290+0000 7f03fe59c640 1 --2- 172.21.15.64:0/1802122058 >> v2:172.21.15.162:3300/0 conn(0x7f0400072230 0x7f0400150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.290+0000 7f03fd59a640 1 -- 172.21.15.64:0/1802122058 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432810481 0 0) 0x7f0400071190 con 0x7f04000cf6d0 2023-11-24T06:28:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.290+0000 7f03fd59a640 1 -- 172.21.15.64:0/1802122058 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e8003520 con 0x7f04000cf6d0 2023-11-24T06:28:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.291+0000 7f03fdd9b640 1 -- 172.21.15.64:0/1802122058 >> v2:172.21.15.162:3300/0 conn(0x7f0400072230 msgr2=0x7f0400150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.291+0000 7f03fdd9b640 1 --2- 172.21.15.64:0/1802122058 >> v2:172.21.15.162:3300/0 conn(0x7f0400072230 0x7f0400150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.291+0000 7f03fdd9b640 1 -- 172.21.15.64:0/1802122058 >> v1:172.21.15.64:6789/0 conn(0x7f04000cf6d0 legacy=0x7f04000b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.291+0000 7f03fdd9b640 1 -- 172.21.15.64:0/1802122058 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0400071750 con 0x7f04000cf290 2023-11-24T06:28:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.292+0000 7f03fdd9b640 1 --2- 172.21.15.64:0/1802122058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04000cf290 0x7f040015d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03f40097a0 tx=0x7f03f402dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a63cdff5369ab8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.292+0000 7f03fd59a640 1 -- 172.21.15.64:0/1802122058 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f402f880 con 0x7f04000cf290 2023-11-24T06:28:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f03fd59a640 1 -- 172.21.15.64:0/1802122058 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f402f9e0 con 0x7f04000cf290 2023-11-24T06:28:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f03fd59a640 1 -- 172.21.15.64:0/1802122058 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f402fd70 con 0x7f04000cf290 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f0404a31640 1 -- 172.21.15.64:0/1802122058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04000cf290 msgr2=0x7f040015d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f0404a31640 1 --2- 172.21.15.64:0/1802122058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04000cf290 0x7f040015d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03f40097a0 tx=0x7f03f402dac0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f0404a31640 1 -- 172.21.15.64:0/1802122058 shutdown_connections 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f0404a31640 1 --2- 172.21.15.64:0/1802122058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04000cf290 0x7f040015d300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f0404a31640 1 --2- 172.21.15.64:0/1802122058 >> v2:172.21.15.162:3300/0 conn(0x7f0400072230 0x7f0400150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f0404a31640 1 -- 172.21.15.64:0/1802122058 >> 172.21.15.64:0/1802122058 conn(0x7f040006a800 msgr2=0x7f040006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.293+0000 7f0404a31640 1 -- 172.21.15.64:0/1802122058 shutdown_connections 2023-11-24T06:28:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.294+0000 7f0404a31640 1 -- 172.21.15.64:0/1802122058 wait complete. 2023-11-24T06:28:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.295+0000 7f0404a31640 1 Processor -- start 2023-11-24T06:28:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.295+0000 7f0404a31640 1 -- start start 2023-11-24T06:28:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.296+0000 7f0404a31640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0400072230 0x7f0400144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.296+0000 7f0404a31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000cf290 0x7f04001579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.296+0000 7f0404a31640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000cf6d0 0x7f04001454b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.296+0000 7f0404a31640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0400153380 con 0x7f04000cf290 2023-11-24T06:28:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.296+0000 7f0404a31640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0400145230 con 0x7f04000cf6d0 2023-11-24T06:28:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.296+0000 7f0404a31640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0400152dc0 con 0x7f0400072230 2023-11-24T06:28:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000cf290 0x7f04001579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000cf290 0x7f04001579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41758/0 (socket says 172.21.15.64:41758) 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 -- 172.21.15.64:0/4157069908 learned_addr learned my addr 172.21.15.64:0/4157069908 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fe59c640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0400072230 0x7f0400144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fed9d640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000cf6d0 0x7f04001454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 -- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0400072230 msgr2=0x7f0400144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0400072230 0x7f0400144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 -- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000cf6d0 msgr2=0x7f04001454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000cf6d0 0x7f04001454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 -- 172.21.15.64:0/4157069908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009400 con 0x7f04000cf290 2023-11-24T06:28:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fed9d640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000cf6d0 0x7f04001454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.297+0000 7f03fdd9b640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000cf290 0x7f04001579c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f03f402e0d0 tx=0x7f03f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.298+0000 7f03eeffd640 1 -- 172.21.15.64:0/4157069908 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f403e040 con 0x7f04000cf290 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.298+0000 7f03eeffd640 1 -- 172.21.15.64:0/4157069908 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4030cb0 con 0x7f04000cf290 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.298+0000 7f03eeffd640 1 -- 172.21.15.64:0/4157069908 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4039640 con 0x7f04000cf290 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.298+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04001530a0 con 0x7f04000cf290 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.298+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0400159ac0 con 0x7f04000cf290 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.299+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0400159640 con 0x7f04000cf290 2023-11-24T06:28:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.300+0000 7f03eeffd640 1 -- 172.21.15.64:0/4157069908 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f03f40397a0 con 0x7f04000cf290 2023-11-24T06:28:22.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.301+0000 7f03eeffd640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03e006ae90 0x7f03e006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.301+0000 7f03eeffd640 1 -- 172.21.15.64:0/4157069908 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03f40ad5c0 con 0x7f04000cf290 2023-11-24T06:28:22.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.301+0000 7f03fe59c640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03e006ae90 0x7f03e006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.302+0000 7f03fe59c640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03e006ae90 0x7f03e006d350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009ad0 tx=0x7f03f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:22.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.305+0000 7f03eeffd640 1 -- 172.21.15.64:0/4157069908 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0400159640 con 0x7f04000cf290 2023-11-24T06:28:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.506+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0400159640 con 0x7f04000cf290 2023-11-24T06:28:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.507+0000 7f03eeffd640 1 -- 172.21.15.64:0/4157069908 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0400159640 con 0x7f04000cf290 2023-11-24T06:28:22.510 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03e006ae90 msgr2=0x7f03e006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03e006ae90 0x7f03e006d350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009ad0 tx=0x7f03f0008040 comp rx=0 tx=0).stop 2023-11-24T06:28:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000cf290 msgr2=0x7f04001579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000cf290 0x7f04001579c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f03f402e0d0 tx=0x7f03f4004730 comp rx=0 tx=0).stop 2023-11-24T06:28:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 shutdown_connections 2023-11-24T06:28:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03e006ae90 0x7f03e006d350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04000cf6d0 0x7f04001454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000cf290 0x7f04001579c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 --2- 172.21.15.64:0/4157069908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0400072230 0x7f0400144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 >> 172.21.15.64:0/4157069908 conn(0x7f040006a800 msgr2=0x7f040006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 shutdown_connections 2023-11-24T06:28:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.510+0000 7f0404a31640 1 -- 172.21.15.64:0/4157069908 wait complete. 2023-11-24T06:28:22.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:22.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.760+0000 7f5f76c46640 1 Processor -- start 2023-11-24T06:28:22.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.761+0000 7f5f76c46640 1 -- start start 2023-11-24T06:28:22.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.762+0000 7f5f76c46640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f7013df80 0x7f5f7015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.762+0000 7f5f76c46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013e360 0x7f5f700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.763+0000 7f5f76c46640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7006c470 con 0x7f5f7015f9c0 2023-11-24T06:28:22.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.763+0000 7f5f76c46640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7006c750 con 0x7f5f7013df80 2023-11-24T06:28:22.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.763+0000 7f5f76c46640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7006ca30 con 0x7f5f7013e360 2023-11-24T06:28:22.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.763+0000 7f5f749e2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f7013df80 0x7f5f7015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.763+0000 7f5f751e3640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5f7015f9c0 0x7f5f701408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58228/0 (socket says 172.21.15.64:58228) 2023-11-24T06:28:22.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.763+0000 7f5f751e3640 1 -- 172.21.15.64:0/4038100959 learned_addr learned my addr 172.21.15.64:0/4038100959 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:22.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.763+0000 7f5f67fff640 1 --2- 172.21.15.64:0/4038100959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013e360 0x7f5f700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.764+0000 7f5f749e2640 1 -- 172.21.15.64:0/4038100959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013e360 msgr2=0x7f5f700cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:22.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.764+0000 7f5f749e2640 1 --2- 172.21.15.64:0/4038100959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013e360 0x7f5f700cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.764+0000 7f5f749e2640 1 -- 172.21.15.64:0/4038100959 >> v1:172.21.15.64:6789/0 conn(0x7f5f7015f9c0 legacy=0x7f5f701408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.764+0000 7f5f749e2640 1 -- 172.21.15.64:0/4038100959 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7006cd10 con 0x7f5f7013df80 2023-11-24T06:28:22.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.764+0000 7f5f677fe640 1 -- 172.21.15.64:0/4038100959 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451512663 0 0) 0x7f5f7006c470 con 0x7f5f7015f9c0 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.765+0000 7f5f749e2640 1 --2- 172.21.15.64:0/4038100959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013df80 0x7f5f7015bee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009d90 tx=0x7f5f6c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7823427bc5386f52 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.765+0000 7f5f677fe640 1 -- 172.21.15.64:0/4038100959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c02cc50 con 0x7f5f7013df80 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.765+0000 7f5f677fe640 1 -- 172.21.15.64:0/4038100959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c02cdb0 con 0x7f5f7013df80 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.765+0000 7f5f677fe640 1 -- 172.21.15.64:0/4038100959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c034660 con 0x7f5f7013df80 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 -- 172.21.15.64:0/4038100959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013df80 msgr2=0x7f5f7015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 --2- 172.21.15.64:0/4038100959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013df80 0x7f5f7015bee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009d90 tx=0x7f5f6c02bea0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 -- 172.21.15.64:0/4038100959 shutdown_connections 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 --2- 172.21.15.64:0/4038100959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013e360 0x7f5f700cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 --2- 172.21.15.64:0/4038100959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013df80 0x7f5f7015bee0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 -- 172.21.15.64:0/4038100959 >> 172.21.15.64:0/4038100959 conn(0x7f5f7006a800 msgr2=0x7f5f7006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:22.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 -- 172.21.15.64:0/4038100959 shutdown_connections 2023-11-24T06:28:22.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.766+0000 7f5f76c46640 1 -- 172.21.15.64:0/4038100959 wait complete. 2023-11-24T06:28:22.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.768+0000 7f5f76c46640 1 Processor -- start 2023-11-24T06:28:22.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.768+0000 7f5f76c46640 1 -- start start 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.769+0000 7f5f76c46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013df80 0x7f5f7014f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.769+0000 7f5f76c46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013e360 0x7f5f701f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.769+0000 7f5f76c46640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7015f9c0 0x7f5f70151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.769+0000 7f5f76c46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b1ca0 con 0x7f5f7013e360 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.769+0000 7f5f76c46640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b2820 con 0x7f5f7015f9c0 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.769+0000 7f5f76c46640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b1ef0 con 0x7f5f7013df80 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013e360 0x7f5f701f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013e360 0x7f5f701f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41766/0 (socket says 172.21.15.64:41766) 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 -- 172.21.15.64:0/244821295 learned_addr learned my addr 172.21.15.64:0/244821295 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f749e2640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013df80 0x7f5f7014f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f751e3640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7015f9c0 0x7f5f70151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 -- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013df80 msgr2=0x7f5f7014f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013df80 0x7f5f7014f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 -- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7015f9c0 msgr2=0x7f5f70151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7015f9c0 0x7f5f70151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 -- 172.21.15.64:0/244821295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c009a70 con 0x7f5f7013e360 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f751e3640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7015f9c0 0x7f5f70151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.770+0000 7f5f67fff640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013e360 0x7f5f701f65a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9e0 tx=0x7f5f6000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.771+0000 7f5f657fa640 1 -- 172.21.15.64:0/244821295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60009670 con 0x7f5f7013e360 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.771+0000 7f5f657fa640 1 -- 172.21.15.64:0/244821295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6000c9d0 con 0x7f5f7013e360 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.771+0000 7f5f657fa640 1 -- 172.21.15.64:0/244821295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cd80 con 0x7f5f7013e360 2023-11-24T06:28:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.771+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f700b21d0 con 0x7f5f7013e360 2023-11-24T06:28:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.771+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7014fae0 con 0x7f5f7013e360 2023-11-24T06:28:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.772+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f700675d0 con 0x7f5f7013e360 2023-11-24T06:28:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.773+0000 7f5f657fa640 1 -- 172.21.15.64:0/244821295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f60013470 con 0x7f5f7013e360 2023-11-24T06:28:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.774+0000 7f5f657fa640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f5c06ae40 0x7f5f5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.774+0000 7f5f657fa640 1 -- 172.21.15.64:0/244821295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f60089950 con 0x7f5f7013e360 2023-11-24T06:28:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.774+0000 7f5f749e2640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f5c06ae40 0x7f5f5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.775+0000 7f5f749e2640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f5c06ae40 0x7f5f5c06d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c036000 tx=0x7f5f6c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:22.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.778+0000 7f5f657fa640 1 -- 172.21.15.64:0/244821295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f60054960 con 0x7f5f7013e360 2023-11-24T06:28:22.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.972+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f700b2960 con 0x7f5f7013e360 2023-11-24T06:28:22.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.973+0000 7f5f657fa640 1 -- 172.21.15.64:0/244821295 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f600170a0 con 0x7f5f7013e360 2023-11-24T06:28:22.976 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:22.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f5c06ae40 msgr2=0x7f5f5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f5c06ae40 0x7f5f5c06d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c036000 tx=0x7f5f6c030000 comp rx=0 tx=0).stop 2023-11-24T06:28:22.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013e360 msgr2=0x7f5f701f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013e360 0x7f5f701f65a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9e0 tx=0x7f5f6000eea0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 shutdown_connections 2023-11-24T06:28:22.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f5c06ae40 0x7f5f5c06d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7015f9c0 0x7f5f70151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f7013e360 0x7f5f701f65a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 --2- 172.21.15.64:0/244821295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7013df80 0x7f5f7014f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 >> 172.21.15.64:0/244821295 conn(0x7f5f7006a800 msgr2=0x7f5f700b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:22.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 shutdown_connections 2023-11-24T06:28:22.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:22.976+0000 7f5f76c46640 1 -- 172.21.15.64:0/244821295 wait complete. 2023-11-24T06:28:22.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:23.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.223+0000 7f7c9d3cb640 1 Processor -- start 2023-11-24T06:28:23.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.223+0000 7f7c9d3cb640 1 -- start start 2023-11-24T06:28:23.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.225+0000 7f7c9d3cb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c980cc990 0x7f7c9806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.225+0000 7f7c9d3cb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c98070e50 con 0x7f7c980cc500 2023-11-24T06:28:23.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.225+0000 7f7c9d3cb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c98144020 con 0x7f7c98143c40 2023-11-24T06:28:23.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.225+0000 7f7c9d3cb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c980723c0 con 0x7f7c980cc990 2023-11-24T06:28:23.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.225+0000 7f7c977fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7c980cc500 0x7f7c9806d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34214/0 (socket says 172.21.15.64:34214) 2023-11-24T06:28:23.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.225+0000 7f7c977fe640 1 -- 172.21.15.64:0/3250648811 learned_addr learned my addr 172.21.15.64:0/3250648811 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:23.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.225+0000 7f7c9cbca640 1 --2- 172.21.15.64:0/3250648811 >> v2:172.21.15.64:3300/0 conn(0x7f7c980cc990 0x7f7c9806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.226+0000 7f7c96ffd640 1 -- 172.21.15.64:0/3250648811 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581335732 0 0) 0x7f7c98070e50 con 0x7f7c980cc500 2023-11-24T06:28:23.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.226+0000 7f7c96ffd640 1 -- 172.21.15.64:0/3250648811 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c80003520 con 0x7f7c980cc500 2023-11-24T06:28:23.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.226+0000 7f7c9cbca640 1 -- 172.21.15.64:0/3250648811 >> v1:172.21.15.162:6789/0 conn(0x7f7c98143c40 legacy=0x7f7c98071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:23.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.226+0000 7f7c9cbca640 1 -- 172.21.15.64:0/3250648811 >> v1:172.21.15.64:6789/0 conn(0x7f7c980cc500 legacy=0x7f7c9806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.226+0000 7f7c9cbca640 1 -- 172.21.15.64:0/3250648811 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98071e00 con 0x7f7c980cc990 2023-11-24T06:28:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.227+0000 7f7c9cbca640 1 --2- 172.21.15.64:0/3250648811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c9806d870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7c900097a0 tx=0x7f7c9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce681d9ff144d476 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.227+0000 7f7c96ffd640 1 -- 172.21.15.64:0/3250648811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9002e070 con 0x7f7c980cc990 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.227+0000 7f7c96ffd640 1 -- 172.21.15.64:0/3250648811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c900315a0 con 0x7f7c980cc990 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.227+0000 7f7c96ffd640 1 -- 172.21.15.64:0/3250648811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90031930 con 0x7f7c980cc990 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.228+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/3250648811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 msgr2=0x7f7c9806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.228+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/3250648811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c9806d870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7c900097a0 tx=0x7f7c9002b840 comp rx=0 tx=0).stop 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.228+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/3250648811 shutdown_connections 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.228+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/3250648811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c9806d870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.228+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/3250648811 >> 172.21.15.64:0/3250648811 conn(0x7f7c9806a800 msgr2=0x7f7c9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:23.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.228+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/3250648811 shutdown_connections 2023-11-24T06:28:23.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.228+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/3250648811 wait complete. 2023-11-24T06:28:23.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.230+0000 7f7c9d3cb640 1 Processor -- start 2023-11-24T06:28:23.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.230+0000 7f7c9d3cb640 1 -- start start 2023-11-24T06:28:23.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c9d3cb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c980cc500 0x7f7c9813ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c9d3cb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c98137730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c9d3cb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c98143c40 0x7f7c980a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c9d3cb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98137c70 con 0x7f7c980cc990 2023-11-24T06:28:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c9d3cb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980a87f0 con 0x7f7c98143c40 2023-11-24T06:28:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c9d3cb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98139d00 con 0x7f7c980cc500 2023-11-24T06:28:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c98137730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c98137730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43458/0 (socket says 172.21.15.64:43458) 2023-11-24T06:28:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.231+0000 7f7c977fe640 1 -- 172.21.15.64:0/4009133199 learned_addr learned my addr 172.21.15.64:0/4009133199 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:23.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c97fff640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c980cc500 0x7f7c9813ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c977fe640 1 -- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c980cc500 msgr2=0x7f7c9813ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c977fe640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c980cc500 0x7f7c9813ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c9cbca640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c98143c40 0x7f7c980a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c977fe640 1 -- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c98143c40 msgr2=0x7f7c980a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c977fe640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c98143c40 0x7f7c980a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c977fe640 1 -- 172.21.15.64:0/4009133199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90009400 con 0x7f7c980cc990 2023-11-24T06:28:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c97fff640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c980cc500 0x7f7c9813ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c9cbca640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c98143c40 0x7f7c980a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c977fe640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c98137730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00c500 tx=0x7f7c8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c94ff9640 1 -- 172.21.15.64:0/4009133199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c005ce0 con 0x7f7c980cc990 2023-11-24T06:28:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c94ff9640 1 -- 172.21.15.64:0/4009133199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c005e40 con 0x7f7c980cc990 2023-11-24T06:28:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c94ff9640 1 -- 172.21.15.64:0/4009133199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c007330 con 0x7f7c980cc990 2023-11-24T06:28:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.232+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c98139fe0 con 0x7f7c980cc990 2023-11-24T06:28:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.233+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9813a5a0 con 0x7f7c980cc990 2023-11-24T06:28:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.234+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c98067450 con 0x7f7c980cc990 2023-11-24T06:28:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.235+0000 7f7c94ff9640 1 -- 172.21.15.64:0/4009133199 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c8c007550 con 0x7f7c980cc990 2023-11-24T06:28:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.236+0000 7f7c94ff9640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c8406ae40 0x7f7c8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.236+0000 7f7c94ff9640 1 -- 172.21.15.64:0/4009133199 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c8c088cf0 con 0x7f7c980cc990 2023-11-24T06:28:23.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.236+0000 7f7c97fff640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c8406ae40 0x7f7c8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.237+0000 7f7c97fff640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c8406ae40 0x7f7c8406d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c88007e90 tx=0x7f7c880073a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.240+0000 7f7c94ff9640 1 -- 172.21.15.64:0/4009133199 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c8c053dc0 con 0x7f7c980cc990 2023-11-24T06:28:23.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.434+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c981433f0 con 0x7f7c980cc990 2023-11-24T06:28:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.437+0000 7f7c94ff9640 1 -- 172.21.15.64:0/4009133199 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f7c8c017020 con 0x7f7c980cc990 2023-11-24T06:28:23.441 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:23.441 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c8406ae40 msgr2=0x7f7c8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c8406ae40 0x7f7c8406d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c88007e90 tx=0x7f7c880073a0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 msgr2=0x7f7c98137730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c98137730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00c500 tx=0x7f7c8c00c9c0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 shutdown_connections 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c8406ae40 0x7f7c8406d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c98143c40 0x7f7c980a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c980cc990 0x7f7c98137730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 --2- 172.21.15.64:0/4009133199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c980cc500 0x7f7c9813ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 >> 172.21.15.64:0/4009133199 conn(0x7f7c9806a800 msgr2=0x7f7c9806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 shutdown_connections 2023-11-24T06:28:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.440+0000 7f7c9d3cb640 1 -- 172.21.15.64:0/4009133199 wait complete. 2023-11-24T06:28:23.464 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:23.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:23.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.694+0000 7fea0a6aa640 1 Processor -- start 2023-11-24T06:28:23.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.694+0000 7fea0a6aa640 1 -- start start 2023-11-24T06:28:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.695+0000 7fea0a6aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040cf9a0 0x7fea040cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.695+0000 7fea0a6aa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fea040b31d0 0x7fea040b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.695+0000 7fea0a6aa640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea040d0320 con 0x7fea0415d4b0 2023-11-24T06:28:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.695+0000 7fea0a6aa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea040d1c60 con 0x7fea040b31d0 2023-11-24T06:28:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.695+0000 7fea0a6aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040b2730 con 0x7fea040cf9a0 2023-11-24T06:28:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.696+0000 7fea096a8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fea0415d4b0 0x7fea040d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34230/0 (socket says 172.21.15.64:34230) 2023-11-24T06:28:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.696+0000 7fea096a8640 1 -- 172.21.15.64:0/1395787819 learned_addr learned my addr 172.21.15.64:0/1395787819 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.696+0000 7fea08ea7640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040cf9a0 0x7fea040cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.696+0000 7fea09ea9640 1 --2- 172.21.15.64:0/1395787819 >> v2:172.21.15.162:3300/0 conn(0x7fea040b31d0 0x7fea040b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.696+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764918001 0 0) 0x7fea040d0320 con 0x7fea0415d4b0 2023-11-24T06:28:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.696+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f0003520 con 0x7fea0415d4b0 2023-11-24T06:28:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900309814 0 0) 0x7fe9f0003520 con 0x7fea0415d4b0 2023-11-24T06:28:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040cf9a0 msgr2=0x7fea040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040cf9a0 0x7fea040cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 >> v2:172.21.15.162:3300/0 conn(0x7fea040b31d0 msgr2=0x7fea040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 --2- 172.21.15.64:0/1395787819 >> v2:172.21.15.162:3300/0 conn(0x7fea040b31d0 0x7fea040b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040d1ec0 con 0x7fea0415d4b0 2023-11-24T06:28:23.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fea00001670 con 0x7fea0415d4b0 2023-11-24T06:28:23.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 >> v1:172.21.15.64:6789/0 conn(0x7fea0415d4b0 legacy=0x7fea040d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040cf9a0 0x7fe9f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9f0003ef0 0x7fe9f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fe9f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040d1c60 con 0x7fe9f0004d00 2023-11-24T06:28:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040b2730 con 0x7fe9f0003ef0 2023-11-24T06:28:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f000b980 con 0x7fea040cf9a0 2023-11-24T06:28:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.697+0000 7fea09ea9640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fe9f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.698+0000 7fea09ea9640 1 -- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040cf9a0 msgr2=0x7fe9f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.698+0000 7fea09ea9640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040cf9a0 0x7fe9f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.698+0000 7fea08ea7640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9f0003ef0 0x7fe9f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.698+0000 7fea09ea9640 1 -- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9f0003ef0 msgr2=0x7fe9f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.698+0000 7fea09ea9640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9f0003ef0 0x7fe9f00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.698+0000 7fea09ea9640 1 -- 172.21.15.64:0/1395787819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0003520 con 0x7fe9f0004d00 2023-11-24T06:28:23.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.699+0000 7fea09ea9640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fe9f0005180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009fc0 tx=0x7fe9f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828442b768613f51 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.699+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc02cc40 con 0x7fe9f0004d00 2023-11-24T06:28:23.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.699+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00f040 con 0x7fe9f0004d00 2023-11-24T06:28:23.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.699+0000 7fe9fbfff640 1 -- 172.21.15.64:0/1395787819 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc034440 con 0x7fe9f0004d00 2023-11-24T06:28:23.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.700+0000 7fea0a6aa640 1 -- 172.21.15.64:0/1395787819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 msgr2=0x7fe9f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.700+0000 7fea0a6aa640 1 --2- 172.21.15.64:0/1395787819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fe9f0005180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009fc0 tx=0x7fe9f00057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.700+0000 7fea09ea9640 1 -- 172.21.15.64:0/1395787819 reap_dead start 2023-11-24T06:28:23.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.700+0000 7fea0a6aa640 1 -- 172.21.15.64:0/1395787819 shutdown_connections 2023-11-24T06:28:23.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.700+0000 7fea0a6aa640 1 -- 172.21.15.64:0/1395787819 >> 172.21.15.64:0/1395787819 conn(0x7fea040d3000 msgr2=0x7fea040d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:23.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.700+0000 7fea0a6aa640 1 -- 172.21.15.64:0/1395787819 shutdown_connections 2023-11-24T06:28:23.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.700+0000 7fea0a6aa640 1 -- 172.21.15.64:0/1395787819 wait complete. 2023-11-24T06:28:23.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.702+0000 7fea0a6aa640 1 Processor -- start 2023-11-24T06:28:23.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.702+0000 7fea0a6aa640 1 -- start start 2023-11-24T06:28:23.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea0a6aa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fea040cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea0a6aa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea0415a8c0 0x7fea040cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea0a6aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040afbd0 0x7fea040b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea0a6aa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040cb340 con 0x7fe9f0004d00 2023-11-24T06:28:23.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea0a6aa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040b0590 con 0x7fea0415a8c0 2023-11-24T06:28:23.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea0a6aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040cb8c0 con 0x7fea040afbd0 2023-11-24T06:28:23.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fea040cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fea040cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43478/0 (socket says 172.21.15.64:43478) 2023-11-24T06:28:23.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 -- 172.21.15.64:0/242500757 learned_addr learned my addr 172.21.15.64:0/242500757 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:23.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 -- 172.21.15.64:0/242500757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040afbd0 msgr2=0x7fea040b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040afbd0 0x7fea040b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 -- 172.21.15.64:0/242500757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea0415a8c0 msgr2=0x7fea040cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea0415a8c0 0x7fea040cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.703+0000 7fea096a8640 1 -- 172.21.15.64:0/242500757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040b0f70 con 0x7fe9f0004d00 2023-11-24T06:28:23.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.704+0000 7fea096a8640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fea040cbd50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fea00000f60 tx=0x7fea0000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.704+0000 7fe9f9ffb640 1 -- 172.21.15.64:0/242500757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000cea0 con 0x7fe9f0004d00 2023-11-24T06:28:23.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.704+0000 7fe9f9ffb640 1 -- 172.21.15.64:0/242500757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea00007500 con 0x7fe9f0004d00 2023-11-24T06:28:23.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.704+0000 7fe9f9ffb640 1 -- 172.21.15.64:0/242500757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00004a20 con 0x7fe9f0004d00 2023-11-24T06:28:23.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.704+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea040cacb0 con 0x7fe9f0004d00 2023-11-24T06:28:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.704+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea040cb550 con 0x7fe9f0004d00 2023-11-24T06:28:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.705+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea04067450 con 0x7fe9f0004d00 2023-11-24T06:28:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.706+0000 7fe9f9ffb640 1 -- 172.21.15.64:0/242500757 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea00004b80 con 0x7fe9f0004d00 2023-11-24T06:28:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.707+0000 7fe9f9ffb640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe9e806ae70 0x7fe9e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.707+0000 7fe9f9ffb640 1 -- 172.21.15.64:0/242500757 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea00088950 con 0x7fe9f0004d00 2023-11-24T06:28:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.707+0000 7fea08ea7640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe9e806ae70 0x7fe9e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.708+0000 7fea08ea7640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe9e806ae70 0x7fe9e806d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea040c9630 tx=0x7fe9f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.712+0000 7fe9f9ffb640 1 -- 172.21.15.64:0/242500757 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea000539f0 con 0x7fe9f0004d00 2023-11-24T06:28:23.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.909+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea04073e10 con 0x7fe9f0004d00 2023-11-24T06:28:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.910+0000 7fe9f9ffb640 1 -- 172.21.15.64:0/242500757 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea00018020 con 0x7fe9f0004d00 2023-11-24T06:28:23.913 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe9e806ae70 msgr2=0x7fe9e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe9e806ae70 0x7fe9e806d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea040c9630 tx=0x7fe9f4008040 comp rx=0 tx=0).stop 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 msgr2=0x7fea040cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fea040cbd50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fea00000f60 tx=0x7fea0000cc50 comp rx=0 tx=0).stop 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 shutdown_connections 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe9e806ae70 0x7fe9e806d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea040afbd0 0x7fea040b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea0415a8c0 0x7fea040cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 --2- 172.21.15.64:0/242500757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9f0004d00 0x7fea040cbd50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 >> 172.21.15.64:0/242500757 conn(0x7fea040d3000 msgr2=0x7fea040b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 shutdown_connections 2023-11-24T06:28:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:23.913+0000 7fea0a6aa640 1 -- 172.21.15.64:0/242500757 wait complete. 2023-11-24T06:28:23.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:24.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.163+0000 7fc9852e1640 1 Processor -- start 2023-11-24T06:28:24.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.163+0000 7fc9852e1640 1 -- start start 2023-11-24T06:28:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.165+0000 7fc9852e1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc9800ccb50 0x7fc980070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.165+0000 7fc9852e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 0x7fc980143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.165+0000 7fc9852e1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98015f4e0 con 0x7fc9800adf20 2023-11-24T06:28:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.165+0000 7fc9852e1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98015f970 con 0x7fc9800ccb50 2023-11-24T06:28:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.165+0000 7fc9852e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98015c3c0 con 0x7fc98015bf10 2023-11-24T06:28:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.165+0000 7fc97ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc9800adf20 0x7fc9800cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34242/0 (socket says 172.21.15.64:34242) 2023-11-24T06:28:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97ffff640 1 -- 172.21.15.64:0/696611794 learned_addr learned my addr 172.21.15.64:0/696611794 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:24.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034080441 0 0) 0x7fc98015f4e0 con 0x7fc9800adf20 2023-11-24T06:28:24.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc968003520 con 0x7fc9800adf20 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97f7fe640 1 --2- 172.21.15.64:0/696611794 >> v2:172.21.15.162:3300/0 conn(0x7fc9800ccb50 0x7fc980070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc984ae0640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 0x7fc980143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 176444591 0 0) 0x7fc968003520 con 0x7fc9800adf20 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 msgr2=0x7fc980143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 0x7fc980143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 >> v2:172.21.15.162:3300/0 conn(0x7fc9800ccb50 msgr2=0x7fc980070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 --2- 172.21.15.64:0/696611794 >> v2:172.21.15.162:3300/0 conn(0x7fc9800ccb50 0x7fc980070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980071130 con 0x7fc9800adf20 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fc970001670 con 0x7fc9800adf20 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 >> v1:172.21.15.64:6789/0 conn(0x7fc9800adf20 legacy=0x7fc9800cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.166+0000 7fc97effd640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 0x7fc968003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc97effd640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc968003ff0 0x7fc9680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc97effd640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc968005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98015f970 con 0x7fc968004d00 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98015c3c0 con 0x7fc968003ff0 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96800bb90 con 0x7fc98015bf10 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc984ae0640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc968005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc97f7fe640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc968003ff0 0x7fc9680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc97ffff640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 0x7fc968003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc984ae0640 1 -- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 msgr2=0x7fc968003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc984ae0640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc98015bf10 0x7fc968003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc984ae0640 1 -- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc968003ff0 msgr2=0x7fc9680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.167+0000 7fc984ae0640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc968003ff0 0x7fc9680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.168+0000 7fc984ae0640 1 -- 172.21.15.64:0/696611794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968003520 con 0x7fc968004d00 2023-11-24T06:28:24.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.169+0000 7fc984ae0640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc968005180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc978007360 tx=0x7fc9680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c08e5802805843e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.169+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97802ccd0 con 0x7fc968004d00 2023-11-24T06:28:24.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.169+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97802ce30 con 0x7fc968004d00 2023-11-24T06:28:24.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.169+0000 7fc97effd640 1 -- 172.21.15.64:0/696611794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978034660 con 0x7fc968004d00 2023-11-24T06:28:24.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.170+0000 7fc9852e1640 1 -- 172.21.15.64:0/696611794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 msgr2=0x7fc968005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.170+0000 7fc9852e1640 1 --2- 172.21.15.64:0/696611794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc968005180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc978007360 tx=0x7fc9680057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.170+0000 7fc984ae0640 1 -- 172.21.15.64:0/696611794 reap_dead start 2023-11-24T06:28:24.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.170+0000 7fc9852e1640 1 -- 172.21.15.64:0/696611794 shutdown_connections 2023-11-24T06:28:24.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.170+0000 7fc9852e1640 1 -- 172.21.15.64:0/696611794 >> 172.21.15.64:0/696611794 conn(0x7fc98006a7d0 msgr2=0x7fc98006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.170+0000 7fc9852e1640 1 -- 172.21.15.64:0/696611794 shutdown_connections 2023-11-24T06:28:24.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.170+0000 7fc9852e1640 1 -- 172.21.15.64:0/696611794 wait complete. 2023-11-24T06:28:24.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.172+0000 7fc9852e1640 1 Processor -- start 2023-11-24T06:28:24.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.173+0000 7fc9852e1640 1 -- start start 2023-11-24T06:28:24.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc9852e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc9801411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc9852e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9800c3d00 0x7fc9800c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc9852e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9800afb80 0x7fc9800b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc9852e1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800b0570 con 0x7fc968004d00 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc9852e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801f07d0 con 0x7fc9800afb80 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc9852e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800ab870 con 0x7fc9800c3d00 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc97ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc9801411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc97ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc9801411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43488/0 (socket says 172.21.15.64:43488) 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.174+0000 7fc97ffff640 1 -- 172.21.15.64:0/632701937 learned_addr learned my addr 172.21.15.64:0/632701937 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97ffff640 1 -- 172.21.15.64:0/632701937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9800c3d00 msgr2=0x7fc9800c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97ffff640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9800c3d00 0x7fc9800c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97ffff640 1 -- 172.21.15.64:0/632701937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9800afb80 msgr2=0x7fc9800b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97ffff640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9800afb80 0x7fc9800b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97ffff640 1 -- 172.21.15.64:0/632701937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800abb50 con 0x7fc968004d00 2023-11-24T06:28:24.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97ffff640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc9801411a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e870 tx=0x7fc97000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97cff9640 1 -- 172.21.15.64:0/632701937 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97000c850 con 0x7fc968004d00 2023-11-24T06:28:24.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97cff9640 1 -- 172.21.15.64:0/632701937 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97000c9b0 con 0x7fc968004d00 2023-11-24T06:28:24.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc97cff9640 1 -- 172.21.15.64:0/632701937 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97000cd60 con 0x7fc968004d00 2023-11-24T06:28:24.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.175+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9800abe30 con 0x7fc968004d00 2023-11-24T06:28:24.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.176+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9800ac3f0 con 0x7fc968004d00 2023-11-24T06:28:24.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.177+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc980066eb0 con 0x7fc968004d00 2023-11-24T06:28:24.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.178+0000 7fc97cff9640 1 -- 172.21.15.64:0/632701937 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc97001d460 con 0x7fc968004d00 2023-11-24T06:28:24.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.179+0000 7fc97cff9640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc96c06ae40 0x7fc96c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.179+0000 7fc97cff9640 1 -- 172.21.15.64:0/632701937 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc970089570 con 0x7fc968004d00 2023-11-24T06:28:24.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.179+0000 7fc97f7fe640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc96c06ae40 0x7fc96c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.180+0000 7fc97f7fe640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc96c06ae40 0x7fc96c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc974006b10 tx=0x7fc974006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.183+0000 7fc97cff9640 1 -- 172.21.15.64:0/632701937 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc970054620 con 0x7fc968004d00 2023-11-24T06:28:24.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.369+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc980067450 con 0x7fc968004d00 2023-11-24T06:28:24.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.370+0000 7fc97cff9640 1 -- 172.21.15.64:0/632701937 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc97005e070 con 0x7fc968004d00 2023-11-24T06:28:24.373 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc96c06ae40 msgr2=0x7fc96c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc96c06ae40 0x7fc96c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc974006b10 tx=0x7fc974006a20 comp rx=0 tx=0).stop 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 msgr2=0x7fc9801411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc9801411a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e870 tx=0x7fc97000ed30 comp rx=0 tx=0).stop 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 shutdown_connections 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc96c06ae40 0x7fc96c06d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9800afb80 0x7fc9800b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9800c3d00 0x7fc9800c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 --2- 172.21.15.64:0/632701937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc968004d00 0x7fc9801411a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 >> 172.21.15.64:0/632701937 conn(0x7fc98006a7d0 msgr2=0x7fc9800cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 shutdown_connections 2023-11-24T06:28:24.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.373+0000 7fc9852e1640 1 -- 172.21.15.64:0/632701937 wait complete. 2023-11-24T06:28:24.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.614+0000 7f293359e640 1 Processor -- start 2023-11-24T06:28:24.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.614+0000 7f293359e640 1 -- start start 2023-11-24T06:28:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.615+0000 7f293359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 0x7f29340d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.615+0000 7f293359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29340b41e0 0x7f29340cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.615+0000 7f293359e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29340d0320 con 0x7f29340cf9a0 2023-11-24T06:28:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.615+0000 7f293359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29340b2730 con 0x7f29340b41e0 2023-11-24T06:28:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.615+0000 7f293359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340d1ec0 con 0x7f293415d4b0 2023-11-24T06:28:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f2931d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f29340cf9a0 0x7f29340cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34252/0 (socket says 172.21.15.64:34252) 2023-11-24T06:28:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f2931d9b640 1 -- 172.21.15.64:0/862464088 learned_addr learned my addr 172.21.15.64:0/862464088 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f2932d9d640 1 --2- 172.21.15.64:0/862464088 >> v2:172.21.15.64:3300/0 conn(0x7f29340b41e0 0x7f29340cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f293259c640 1 --2- 172.21.15.64:0/862464088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 0x7f29340d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f293159a640 1 -- 172.21.15.64:0/862464088 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257506853 0 0) 0x7f29340d0320 con 0x7f29340cf9a0 2023-11-24T06:28:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f293159a640 1 -- 172.21.15.64:0/862464088 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f291c003520 con 0x7f29340cf9a0 2023-11-24T06:28:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f2932d9d640 1 -- 172.21.15.64:0/862464088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 msgr2=0x7f29340d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f2932d9d640 1 --2- 172.21.15.64:0/862464088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 0x7f29340d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f2932d9d640 1 -- 172.21.15.64:0/862464088 >> v1:172.21.15.64:6789/0 conn(0x7f29340cf9a0 legacy=0x7f29340cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.616+0000 7f2932d9d640 1 -- 172.21.15.64:0/862464088 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340b29d0 con 0x7f29340b41e0 2023-11-24T06:28:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.617+0000 7f2932d9d640 1 --2- 172.21.15.64:0/862464088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340b41e0 0x7f29340cfde0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f292c009dc0 tx=0x7f292c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bc560ad47b45bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293159a640 1 -- 172.21.15.64:0/862464088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c02cc40 con 0x7f29340b41e0 2023-11-24T06:28:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293159a640 1 -- 172.21.15.64:0/862464088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c02cda0 con 0x7f29340b41e0 2023-11-24T06:28:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293159a640 1 -- 172.21.15.64:0/862464088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c034660 con 0x7f29340b41e0 2023-11-24T06:28:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 -- 172.21.15.64:0/862464088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340b41e0 msgr2=0x7f29340cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 --2- 172.21.15.64:0/862464088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340b41e0 0x7f29340cfde0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f292c009dc0 tx=0x7f292c02be90 comp rx=0 tx=0).stop 2023-11-24T06:28:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 -- 172.21.15.64:0/862464088 shutdown_connections 2023-11-24T06:28:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 --2- 172.21.15.64:0/862464088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340b41e0 0x7f29340cfde0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 --2- 172.21.15.64:0/862464088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 0x7f29340d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 -- 172.21.15.64:0/862464088 >> 172.21.15.64:0/862464088 conn(0x7f29340d3000 msgr2=0x7f29340d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 -- 172.21.15.64:0/862464088 shutdown_connections 2023-11-24T06:28:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.618+0000 7f293359e640 1 -- 172.21.15.64:0/862464088 wait complete. 2023-11-24T06:28:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.620+0000 7f293359e640 1 Processor -- start 2023-11-24T06:28:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.620+0000 7f293359e640 1 -- start start 2023-11-24T06:28:24.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.621+0000 7f293359e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29340b41e0 0x7f29341f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.621+0000 7f293359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340cf9a0 0x7f29340b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.621+0000 7f293359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 0x7f2934160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.621+0000 7f293359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29341f7240 con 0x7f29340cf9a0 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.621+0000 7f293359e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340b0ed0 con 0x7f29340b41e0 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.621+0000 7f293359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29341467b0 con 0x7f293415d4b0 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340cf9a0 0x7f29340b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340cf9a0 0x7f29340b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43496/0 (socket says 172.21.15.64:43496) 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 -- 172.21.15.64:0/3146990258 learned_addr learned my addr 172.21.15.64:0/3146990258 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f293259c640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29340b41e0 0x7f29341f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 -- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 msgr2=0x7f2934160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 0x7f2934160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 -- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29340b41e0 msgr2=0x7f29341f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29340b41e0 0x7f29341f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 -- 172.21.15.64:0/3146990258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c009a70 con 0x7f29340cf9a0 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f293259c640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29340b41e0 0x7f29341f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.622+0000 7f2931d9b640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340cf9a0 0x7f29340b0990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f292800c500 tx=0x7f292800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.623+0000 7f2922ffd640 1 -- 172.21.15.64:0/3146990258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928005ce0 con 0x7f29340cf9a0 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.623+0000 7f2922ffd640 1 -- 172.21.15.64:0/3146990258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2928005e40 con 0x7f29340cf9a0 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.623+0000 7f2922ffd640 1 -- 172.21.15.64:0/3146990258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928007330 con 0x7f29340cf9a0 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.623+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293415da10 con 0x7f29340cf9a0 2023-11-24T06:28:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.623+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29341f6c80 con 0x7f29340cf9a0 2023-11-24T06:28:24.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.624+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2934146550 con 0x7f29340cf9a0 2023-11-24T06:28:24.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.625+0000 7f2922ffd640 1 -- 172.21.15.64:0/3146990258 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2928007490 con 0x7f29340cf9a0 2023-11-24T06:28:24.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.626+0000 7f2922ffd640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f291406ae70 0x7f291406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.626+0000 7f2922ffd640 1 -- 172.21.15.64:0/3146990258 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2928088bf0 con 0x7f29340cf9a0 2023-11-24T06:28:24.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.626+0000 7f293259c640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f291406ae70 0x7f291406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.627+0000 7f293259c640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f291406ae70 0x7f291406d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2924006fd0 tx=0x7f2924008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.630+0000 7f2922ffd640 1 -- 172.21.15.64:0/3146990258 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2928053d50 con 0x7f29340cf9a0 2023-11-24T06:28:24.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.833+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2934067450 con 0x7f29340cf9a0 2023-11-24T06:28:24.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.834+0000 7f2922ffd640 1 -- 172.21.15.64:0/3146990258 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2928017020 con 0x7f29340cf9a0 2023-11-24T06:28:24.837 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:24.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f291406ae70 msgr2=0x7f291406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f291406ae70 0x7f291406d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2924006fd0 tx=0x7f2924008040 comp rx=0 tx=0).stop 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340cf9a0 msgr2=0x7f29340b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340cf9a0 0x7f29340b0990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f292800c500 tx=0x7f292800c9c0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 shutdown_connections 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f291406ae70 0x7f291406d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f293415d4b0 0x7f2934160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29340cf9a0 0x7f29340b0990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 --2- 172.21.15.64:0/3146990258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29340b41e0 0x7f29341f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 >> 172.21.15.64:0/3146990258 conn(0x7f29340d3000 msgr2=0x7f29340d36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 shutdown_connections 2023-11-24T06:28:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:24.837+0000 7f293359e640 1 -- 172.21.15.64:0/3146990258 wait complete. 2023-11-24T06:28:24.968 INFO:tasks.cephfs_test_runner:Starting test: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T06:28:24.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:25.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.205+0000 7ffa73381640 1 Processor -- start 2023-11-24T06:28:25.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.205+0000 7ffa73381640 1 -- start start 2023-11-24T06:28:25.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.206+0000 7ffa73381640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffa6c06d760 0x7ffa6c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.206+0000 7ffa73381640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c071790 con 0x7ffa6c0b3c20 2023-11-24T06:28:25.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.206+0000 7ffa73381640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c071100 con 0x7ffa6c159590 2023-11-24T06:28:25.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.206+0000 7ffa73381640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6c070860 con 0x7ffa6c06d760 2023-11-24T06:28:25.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.207+0000 7ffa71b7e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa6c0b3c20 0x7ffa6c06d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34254/0 (socket says 172.21.15.64:34254) 2023-11-24T06:28:25.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.207+0000 7ffa71b7e640 1 -- 172.21.15.64:0/3794565002 learned_addr learned my addr 172.21.15.64:0/3794565002 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:25.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.207+0000 7ffa72b80640 1 --2- 172.21.15.64:0/3794565002 >> v2:172.21.15.162:3300/0 conn(0x7ffa6c06d760 0x7ffa6c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.207+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834734295 0 0) 0x7ffa6c071790 con 0x7ffa6c0b3c20 2023-11-24T06:28:25.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.207+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58003520 con 0x7ffa6c0b3c20 2023-11-24T06:28:25.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2247602840 0 0) 0x7ffa58003520 con 0x7ffa6c0b3c20 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 >> v2:172.21.15.162:3300/0 conn(0x7ffa6c06d760 msgr2=0x7ffa6c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 --2- 172.21.15.64:0/3794565002 >> v2:172.21.15.162:3300/0 conn(0x7ffa6c06d760 0x7ffa6c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 >> v1:172.21.15.162:6789/0 conn(0x7ffa6c159590 legacy=0x7ffa6c0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c070b40 con 0x7ffa6c0b3c20 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7ffa5c001690 con 0x7ffa6c0b3c20 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 >> v1:172.21.15.64:6789/0 conn(0x7ffa6c0b3c20 legacy=0x7ffa6c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa580039c0 0x7ffa58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa580047c0 0x7ffa58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580051d0 0x7ffa5800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c070860 con 0x7ffa580047c0 2023-11-24T06:28:25.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800c120 con 0x7ffa580039c0 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800c2e0 con 0x7ffa580051d0 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa71b7e640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa580047c0 0x7ffa58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.208+0000 7ffa7237f640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa580039c0 0x7ffa58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.209+0000 7ffa72b80640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580051d0 0x7ffa5800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.209+0000 7ffa71b7e640 1 -- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580051d0 msgr2=0x7ffa5800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.209+0000 7ffa71b7e640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580051d0 0x7ffa5800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.209+0000 7ffa71b7e640 1 -- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa580039c0 msgr2=0x7ffa58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.209+0000 7ffa71b7e640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa580039c0 0x7ffa58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.209+0000 7ffa71b7e640 1 -- 172.21.15.64:0/3794565002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58003520 con 0x7ffa580047c0 2023-11-24T06:28:25.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.209+0000 7ffa72b80640 1 -- 172.21.15.64:0/3794565002 reap_dead start 2023-11-24T06:28:25.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.210+0000 7ffa71b7e640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa580047c0 0x7ffa58004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c072850 tx=0x7ffa580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc875e246481bdf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.211+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c005ad0 con 0x7ffa580047c0 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.211+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c005c30 con 0x7ffa580047c0 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.211+0000 7ffa7137d640 1 -- 172.21.15.64:0/3794565002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c007070 con 0x7ffa580047c0 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.211+0000 7ffa73381640 1 -- 172.21.15.64:0/3794565002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa580047c0 msgr2=0x7ffa58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.211+0000 7ffa73381640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa580047c0 0x7ffa58004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c072850 tx=0x7ffa580057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.211+0000 7ffa73381640 1 -- 172.21.15.64:0/3794565002 shutdown_connections 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.212+0000 7ffa73381640 1 --2- 172.21.15.64:0/3794565002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa580047c0 0x7ffa58004c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.212+0000 7ffa73381640 1 -- 172.21.15.64:0/3794565002 >> 172.21.15.64:0/3794565002 conn(0x7ffa6c06a800 msgr2=0x7ffa6c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.212+0000 7ffa73381640 1 -- 172.21.15.64:0/3794565002 shutdown_connections 2023-11-24T06:28:25.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.212+0000 7ffa73381640 1 -- 172.21.15.64:0/3794565002 wait complete. 2023-11-24T06:28:25.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.213+0000 7ffa73381640 1 Processor -- start 2023-11-24T06:28:25.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.214+0000 7ffa73381640 1 -- start start 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.214+0000 7ffa73381640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580047c0 0x7ffa6c1e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.214+0000 7ffa73381640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa6c146260 0x7ffa6c1e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa73381640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa6c1e55e0 0x7ffa6c1e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa73381640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c156870 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa73381640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c1466e0 con 0x7ffa6c146260 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa73381640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c146bd0 con 0x7ffa580047c0 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa6c1e55e0 0x7ffa6c1e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa6c1e55e0 0x7ffa6c1e5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43504/0 (socket says 172.21.15.64:43504) 2023-11-24T06:28:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 -- 172.21.15.64:0/1367881910 learned_addr learned my addr 172.21.15.64:0/1367881910 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa7237f640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580047c0 0x7ffa6c1e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa71b7e640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa6c146260 0x7ffa6c1e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 -- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580047c0 msgr2=0x7ffa6c1e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580047c0 0x7ffa6c1e7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 -- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa6c146260 msgr2=0x7ffa6c1e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa6c146260 0x7ffa6c1e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa72b80640 1 -- 172.21.15.64:0/1367881910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c15ca70 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa71b7e640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa6c146260 0x7ffa6c1e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.215+0000 7ffa7237f640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580047c0 0x7ffa6c1e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:25.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.216+0000 7ffa72b80640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa6c1e55e0 0x7ffa6c1e5a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c072820 tx=0x7ffa640115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.216+0000 7ffa62ffd640 1 -- 172.21.15.64:0/1367881910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400e070 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.216+0000 7ffa62ffd640 1 -- 172.21.15.64:0/1367881910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64011de0 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.216+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c159b90 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.216+0000 7ffa62ffd640 1 -- 172.21.15.64:0/1367881910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64003990 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.216+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c15a150 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.217+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c067450 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.219+0000 7ffa62ffd640 1 -- 172.21.15.64:0/1367881910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa64003af0 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.220+0000 7ffa62ffd640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa5006ae60 0x7ffa5006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.220+0000 7ffa62ffd640 1 -- 172.21.15.64:0/1367881910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa64089e40 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.220+0000 7ffa7237f640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa5006ae60 0x7ffa5006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.221+0000 7ffa7237f640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa5006ae60 0x7ffa5006d320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c158e50 tx=0x7ffa68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.224+0000 7ffa62ffd640 1 -- 172.21.15.64:0/1367881910 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa64054e50 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.419+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa6c15ced0 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.421+0000 7ffa62ffd640 1 -- 172.21.15.64:0/1367881910 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7ffa64058470 con 0x7ffa6c1e55e0 2023-11-24T06:28:25.424 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:25.424 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:27:43.346180+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.423+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa5006ae60 msgr2=0x7ffa5006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.423+0000 7ffa73381640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa5006ae60 0x7ffa5006d320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c158e50 tx=0x7ffa68009040 comp rx=0 tx=0).stop 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.423+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa6c1e55e0 msgr2=0x7ffa6c1e5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.423+0000 7ffa73381640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa6c1e55e0 0x7ffa6c1e5a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c072820 tx=0x7ffa640115a0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 shutdown_connections 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa5006ae60 0x7ffa5006d320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa6c1e55e0 0x7ffa6c1e5a60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa6c146260 0x7ffa6c1e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 --2- 172.21.15.64:0/1367881910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa580047c0 0x7ffa6c1e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 >> 172.21.15.64:0/1367881910 conn(0x7ffa6c06a800 msgr2=0x7ffa6c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 shutdown_connections 2023-11-24T06:28:25.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.424+0000 7ffa73381640 1 -- 172.21.15.64:0/1367881910 wait complete. 2023-11-24T06:28:25.445 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:25.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.673+0000 7f6b3d373640 1 Processor -- start 2023-11-24T06:28:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.673+0000 7f6b3d373640 1 -- start start 2023-11-24T06:28:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.675+0000 7f6b3d373640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b380adf20 0x7f6b38070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.675+0000 7f6b3d373640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6b380ccb50 0x7f6b38143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.675+0000 7f6b3d373640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3815f4e0 con 0x7f6b3815bf10 2023-11-24T06:28:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.675+0000 7f6b3d373640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3815f970 con 0x7f6b380adf20 2023-11-24T06:28:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.675+0000 7f6b3d373640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b38071130 con 0x7f6b380ccb50 2023-11-24T06:28:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.675+0000 7f6b37fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b380adf20 0x7f6b38070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.676+0000 7f6b37fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b380adf20 0x7f6b38070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43516/0 (socket says 172.21.15.64:43516) 2023-11-24T06:28:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.676+0000 7f6b37fff640 1 -- 172.21.15.64:0/1058818590 learned_addr learned my addr 172.21.15.64:0/1058818590 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.676+0000 7f6b3cb72640 1 --2- 172.21.15.64:0/1058818590 >> v2:172.21.15.162:3300/0 conn(0x7f6b380ccb50 0x7f6b38143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.676+0000 7f6b37fff640 1 -- 172.21.15.64:0/1058818590 >> v2:172.21.15.162:3300/0 conn(0x7f6b380ccb50 msgr2=0x7f6b38143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.676+0000 7f6b37fff640 1 --2- 172.21.15.64:0/1058818590 >> v2:172.21.15.162:3300/0 conn(0x7f6b380ccb50 0x7f6b38143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.676+0000 7f6b37fff640 1 -- 172.21.15.64:0/1058818590 >> v1:172.21.15.162:6789/0 conn(0x7f6b3815bf10 legacy=0x7f6b380cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.676+0000 7f6b37fff640 1 -- 172.21.15.64:0/1058818590 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3815c390 con 0x7f6b380adf20 2023-11-24T06:28:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.678+0000 7f6b37fff640 1 --2- 172.21.15.64:0/1058818590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b380adf20 0x7f6b38070a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b3806ff20 tx=0x7f6b2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1728d7069813dd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.678+0000 7f6b377fe640 1 -- 172.21.15.64:0/1058818590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280317e0 con 0x7f6b380adf20 2023-11-24T06:28:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.678+0000 7f6b377fe640 1 -- 172.21.15.64:0/1058818590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28031940 con 0x7f6b380adf20 2023-11-24T06:28:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.678+0000 7f6b377fe640 1 -- 172.21.15.64:0/1058818590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28031cd0 con 0x7f6b380adf20 2023-11-24T06:28:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 -- 172.21.15.64:0/1058818590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b380adf20 msgr2=0x7f6b38070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1058818590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b380adf20 0x7f6b38070a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b3806ff20 tx=0x7f6b2802b880 comp rx=0 tx=0).stop 2023-11-24T06:28:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 -- 172.21.15.64:0/1058818590 shutdown_connections 2023-11-24T06:28:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1058818590 >> v2:172.21.15.162:3300/0 conn(0x7f6b380ccb50 0x7f6b38143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1058818590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b380adf20 0x7f6b38070a30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 -- 172.21.15.64:0/1058818590 >> 172.21.15.64:0/1058818590 conn(0x7f6b3806a7d0 msgr2=0x7f6b3806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 -- 172.21.15.64:0/1058818590 shutdown_connections 2023-11-24T06:28:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.679+0000 7f6b3d373640 1 -- 172.21.15.64:0/1058818590 wait complete. 2023-11-24T06:28:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.681+0000 7f6b3d373640 1 Processor -- start 2023-11-24T06:28:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.681+0000 7f6b3d373640 1 -- start start 2023-11-24T06:28:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3d373640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b380adf20 0x7f6b3813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3d373640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b380ccb50 0x7f6b3815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3d373640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3815bf10 0x7f6b380b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3d373640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3813e6b0 con 0x7f6b3815bf10 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3d373640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3815e160 con 0x7f6b380adf20 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3d373640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3815e440 con 0x7f6b380ccb50 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3815bf10 0x7f6b380b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3815bf10 0x7f6b380b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43520/0 (socket says 172.21.15.64:43520) 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b3cb72640 1 -- 172.21.15.64:0/1092985327 learned_addr learned my addr 172.21.15.64:0/1092985327 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.682+0000 7f6b2ffff640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b380ccb50 0x7f6b3815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b37fff640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b380adf20 0x7f6b3813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b3cb72640 1 -- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b380ccb50 msgr2=0x7f6b3815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b3cb72640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b380ccb50 0x7f6b3815d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b3cb72640 1 -- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b380adf20 msgr2=0x7f6b3813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b3cb72640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b380adf20 0x7f6b3813ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b3cb72640 1 -- 172.21.15.64:0/1092985327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280094d0 con 0x7f6b3815bf10 2023-11-24T06:28:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b3cb72640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3815bf10 0x7f6b380b3150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000cd70 tx=0x7f6b3000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b357fa640 1 -- 172.21.15.64:0/1092985327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30003b90 con 0x7f6b3815bf10 2023-11-24T06:28:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b357fa640 1 -- 172.21.15.64:0/1092985327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30003cf0 con 0x7f6b3815bf10 2023-11-24T06:28:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b357fa640 1 -- 172.21.15.64:0/1092985327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30004240 con 0x7f6b3815bf10 2023-11-24T06:28:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.683+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3813dff0 con 0x7f6b3815bf10 2023-11-24T06:28:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.684+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3815dd50 con 0x7f6b3815bf10 2023-11-24T06:28:25.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.685+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3813e7f0 con 0x7f6b3815bf10 2023-11-24T06:28:25.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.686+0000 7f6b357fa640 1 -- 172.21.15.64:0/1092985327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b300043a0 con 0x7f6b3815bf10 2023-11-24T06:28:25.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.687+0000 7f6b357fa640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b1806ae40 0x7f6b1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.688+0000 7f6b357fa640 1 -- 172.21.15.64:0/1092985327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b30012070 con 0x7f6b3815bf10 2023-11-24T06:28:25.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.688+0000 7f6b37fff640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b1806ae40 0x7f6b1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.688+0000 7f6b37fff640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b1806ae40 0x7f6b1806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b28036000 tx=0x7f6b2803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.693+0000 7f6b357fa640 1 -- 172.21.15.64:0/1092985327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b30055060 con 0x7f6b3815bf10 2023-11-24T06:28:25.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:25.884 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:25.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.886+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b38067450 con 0x7f6b3815bf10 2023-11-24T06:28:25.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.887+0000 7f6b357fa640 1 -- 172.21.15.64:0/1092985327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b30058680 con 0x7f6b3815bf10 2023-11-24T06:28:25.889 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:25.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b1806ae40 msgr2=0x7f6b1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b1806ae40 0x7f6b1806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b28036000 tx=0x7f6b2803d040 comp rx=0 tx=0).stop 2023-11-24T06:28:25.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3815bf10 msgr2=0x7f6b380b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3815bf10 0x7f6b380b3150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000cd70 tx=0x7f6b3000a660 comp rx=0 tx=0).stop 2023-11-24T06:28:25.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 shutdown_connections 2023-11-24T06:28:25.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b1806ae40 0x7f6b1806d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3815bf10 0x7f6b380b3150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b380ccb50 0x7f6b3815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 --2- 172.21.15.64:0/1092985327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b380adf20 0x7f6b3813ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 >> 172.21.15.64:0/1092985327 conn(0x7f6b3806a7d0 msgr2=0x7f6b380ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 shutdown_connections 2023-11-24T06:28:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:25.889+0000 7f6b3d373640 1 -- 172.21.15.64:0/1092985327 wait complete. 2023-11-24T06:28:25.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.173+0000 7fbda9e9b640 1 Processor -- start 2023-11-24T06:28:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.173+0000 7fbda9e9b640 1 -- start start 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.175+0000 7fbda9e9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda40cf9a0 0x7fbda4070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.175+0000 7fbda9e9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbda4070ea0 0x7fbda40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.175+0000 7fbda9e9b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda4071690 con 0x7fbda414b2c0 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.175+0000 7fbda9e9b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda414b6a0 con 0x7fbda4070ea0 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.175+0000 7fbda9e9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4071970 con 0x7fbda40cf9a0 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.175+0000 7fbda3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbda4070ea0 0x7fbda40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.176+0000 7fbda3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbda4070ea0 0x7fbda40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43528/0 (socket says 172.21.15.64:43528) 2023-11-24T06:28:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.176+0000 7fbda3fff640 1 -- 172.21.15.64:0/2426646788 learned_addr learned my addr 172.21.15.64:0/2426646788 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.176+0000 7fbd9affd640 1 --2- 172.21.15.64:0/2426646788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda40cf9a0 0x7fbda4070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.176+0000 7fbda3fff640 1 -- 172.21.15.64:0/2426646788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda40cf9a0 msgr2=0x7fbda4070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.176+0000 7fbda3fff640 1 --2- 172.21.15.64:0/2426646788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda40cf9a0 0x7fbda4070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.176+0000 7fbda3fff640 1 -- 172.21.15.64:0/2426646788 >> v1:172.21.15.162:6789/0 conn(0x7fbda414b2c0 legacy=0x7fbda40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.176+0000 7fbda3fff640 1 -- 172.21.15.64:0/2426646788 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4071c50 con 0x7fbda4070ea0 2023-11-24T06:28:26.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.177+0000 7fbda3fff640 1 --2- 172.21.15.64:0/2426646788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda4070ea0 0x7fbda40cc160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e50 tx=0x7fbd9c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cfb71070b8b031 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda2ffd640 1 -- 172.21.15.64:0/2426646788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c02ca70 con 0x7fbda4070ea0 2023-11-24T06:28:26.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda2ffd640 1 -- 172.21.15.64:0/2426646788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c02cbd0 con 0x7fbda4070ea0 2023-11-24T06:28:26.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda2ffd640 1 -- 172.21.15.64:0/2426646788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0343f0 con 0x7fbda4070ea0 2023-11-24T06:28:26.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 -- 172.21.15.64:0/2426646788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda4070ea0 msgr2=0x7fbda40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/2426646788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda4070ea0 0x7fbda40cc160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e50 tx=0x7fbd9c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 -- 172.21.15.64:0/2426646788 shutdown_connections 2023-11-24T06:28:26.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/2426646788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda4070ea0 0x7fbda40cc160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/2426646788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda40cf9a0 0x7fbda4070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 -- 172.21.15.64:0/2426646788 >> 172.21.15.64:0/2426646788 conn(0x7fbda406a800 msgr2=0x7fbda406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 -- 172.21.15.64:0/2426646788 shutdown_connections 2023-11-24T06:28:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.178+0000 7fbda9e9b640 1 -- 172.21.15.64:0/2426646788 wait complete. 2023-11-24T06:28:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.180+0000 7fbda9e9b640 1 Processor -- start 2023-11-24T06:28:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.180+0000 7fbda9e9b640 1 -- start start 2023-11-24T06:28:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.181+0000 7fbda9e9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda4070ea0 0x7fbda4160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.181+0000 7fbda9e9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda40cf9a0 0x7fbda4148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.181+0000 7fbda9e9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda414b2c0 0x7fbda415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.181+0000 7fbda9e9b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda413f840 con 0x7fbda414b2c0 2023-11-24T06:28:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.181+0000 7fbda9e9b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4146470 con 0x7fbda40cf9a0 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.181+0000 7fbda9e9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4146750 con 0x7fbda4070ea0 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda414b2c0 0x7fbda415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda414b2c0 0x7fbda415f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43542/0 (socket says 172.21.15.64:43542) 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 -- 172.21.15.64:0/1645407679 learned_addr learned my addr 172.21.15.64:0/1645407679 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda37fe640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda4070ea0 0x7fbda4160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbd9affd640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda40cf9a0 0x7fbda4148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 -- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda4070ea0 msgr2=0x7fbda4160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda4070ea0 0x7fbda4160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 -- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda40cf9a0 msgr2=0x7fbda4148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda40cf9a0 0x7fbda4148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda3fff640 1 -- 172.21.15.64:0/1645407679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c009b30 con 0x7fbda414b2c0 2023-11-24T06:28:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.182+0000 7fbda37fe640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda4070ea0 0x7fbda4160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.183+0000 7fbda3fff640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda414b2c0 0x7fbda415f5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e50 tx=0x7fbd9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.183+0000 7fbda0ff9640 1 -- 172.21.15.64:0/1645407679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c009510 con 0x7fbda414b2c0 2023-11-24T06:28:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.183+0000 7fbda0ff9640 1 -- 172.21.15.64:0/1645407679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c03e450 con 0x7fbda414b2c0 2023-11-24T06:28:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.183+0000 7fbda0ff9640 1 -- 172.21.15.64:0/1645407679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c03e800 con 0x7fbda414b2c0 2023-11-24T06:28:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.183+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda4146a30 con 0x7fbda414b2c0 2023-11-24T06:28:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.183+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda4146ff0 con 0x7fbda414b2c0 2023-11-24T06:28:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.184+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda414ab20 con 0x7fbda414b2c0 2023-11-24T06:28:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.185+0000 7fbda0ff9640 1 -- 172.21.15.64:0/1645407679 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd9c03e960 con 0x7fbda414b2c0 2023-11-24T06:28:26.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.186+0000 7fbda0ff9640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbd8406ae40 0x7fbd8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.186+0000 7fbda0ff9640 1 -- 172.21.15.64:0/1645407679 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd9c0ac410 con 0x7fbda414b2c0 2023-11-24T06:28:26.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.187+0000 7fbda37fe640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbd8406ae40 0x7fbd8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.187+0000 7fbda37fe640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbd8406ae40 0x7fbd8406d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006fd0 tx=0x7fbd94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.191+0000 7fbda0ff9640 1 -- 172.21.15.64:0/1645407679 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd9c077420 con 0x7fbda414b2c0 2023-11-24T06:28:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.383+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbda4148690 con 0x7fbda414b2c0 2023-11-24T06:28:26.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.384+0000 7fbda0ff9640 1 -- 172.21.15.64:0/1645407679 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd9c07aa40 con 0x7fbda414b2c0 2023-11-24T06:28:26.386 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbd8406ae40 msgr2=0x7fbd8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbd8406ae40 0x7fbd8406d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006fd0 tx=0x7fbd94008040 comp rx=0 tx=0).stop 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda414b2c0 msgr2=0x7fbda415f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda414b2c0 0x7fbda415f5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e50 tx=0x7fbd9c004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 shutdown_connections 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbd8406ae40 0x7fbd8406d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda414b2c0 0x7fbda415f5e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda40cf9a0 0x7fbda4148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 --2- 172.21.15.64:0/1645407679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbda4070ea0 0x7fbda4160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 >> 172.21.15.64:0/1645407679 conn(0x7fbda406a800 msgr2=0x7fbda406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 shutdown_connections 2023-11-24T06:28:26.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.386+0000 7fbda9e9b640 1 -- 172.21.15.64:0/1645407679 wait complete. 2023-11-24T06:28:26.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:26.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.638+0000 7ffb87cc9640 1 Processor -- start 2023-11-24T06:28:26.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.639+0000 7ffb87cc9640 1 -- start start 2023-11-24T06:28:26.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.640+0000 7ffb87cc9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb8013df80 0x7ffb8015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.640+0000 7ffb87cc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013e360 0x7ffb800cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.640+0000 7ffb87cc9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb8015f9c0 0x7ffb800cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.640+0000 7ffb87cc9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8006c470 con 0x7ffb8015f9c0 2023-11-24T06:28:26.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.640+0000 7ffb87cc9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8006c750 con 0x7ffb8013df80 2023-11-24T06:28:26.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.640+0000 7ffb87cc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8006ca30 con 0x7ffb8013e360 2023-11-24T06:28:26.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb8015f9c0 0x7ffb800cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb8015f9c0 0x7ffb800cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43552/0 (socket says 172.21.15.64:43552) 2023-11-24T06:28:26.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 -- 172.21.15.64:0/822586360 learned_addr learned my addr 172.21.15.64:0/822586360 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:26.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb85264640 1 --2- 172.21.15.64:0/822586360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013e360 0x7ffb800cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb85a65640 1 --2- 172.21.15.64:0/822586360 >> v2:172.21.15.162:3300/0 conn(0x7ffb8013df80 0x7ffb8015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 -- 172.21.15.64:0/822586360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013e360 msgr2=0x7ffb800cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 --2- 172.21.15.64:0/822586360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013e360 0x7ffb800cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 -- 172.21.15.64:0/822586360 >> v2:172.21.15.162:3300/0 conn(0x7ffb8013df80 msgr2=0x7ffb8015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 --2- 172.21.15.64:0/822586360 >> v2:172.21.15.162:3300/0 conn(0x7ffb8013df80 0x7ffb8015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.641+0000 7ffb86266640 1 -- 172.21.15.64:0/822586360 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8006cd10 con 0x7ffb8015f9c0 2023-11-24T06:28:26.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.642+0000 7ffb86266640 1 --2- 172.21.15.64:0/822586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800cf7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffb80072770 tx=0x7ffb7802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc94e763f57c25f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.643+0000 7ffb84a63640 1 -- 172.21.15.64:0/822586360 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78031880 con 0x7ffb8015f9c0 2023-11-24T06:28:26.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.643+0000 7ffb84a63640 1 -- 172.21.15.64:0/822586360 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb780319e0 con 0x7ffb8015f9c0 2023-11-24T06:28:26.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.643+0000 7ffb84a63640 1 -- 172.21.15.64:0/822586360 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78031d70 con 0x7ffb8015f9c0 2023-11-24T06:28:26.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.643+0000 7ffb87cc9640 1 -- 172.21.15.64:0/822586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 msgr2=0x7ffb800cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.643+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/822586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800cf7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffb80072770 tx=0x7ffb7802b920 comp rx=0 tx=0).stop 2023-11-24T06:28:26.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.644+0000 7ffb87cc9640 1 -- 172.21.15.64:0/822586360 shutdown_connections 2023-11-24T06:28:26.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.644+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/822586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800cf7d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.644+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/822586360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013e360 0x7ffb800cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.644+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/822586360 >> v2:172.21.15.162:3300/0 conn(0x7ffb8013df80 0x7ffb8015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.644+0000 7ffb87cc9640 1 -- 172.21.15.64:0/822586360 >> 172.21.15.64:0/822586360 conn(0x7ffb8006a800 msgr2=0x7ffb8006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.644+0000 7ffb87cc9640 1 -- 172.21.15.64:0/822586360 shutdown_connections 2023-11-24T06:28:26.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.644+0000 7ffb87cc9640 1 -- 172.21.15.64:0/822586360 wait complete. 2023-11-24T06:28:26.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.645+0000 7ffb87cc9640 1 Processor -- start 2023-11-24T06:28:26.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.646+0000 7ffb87cc9640 1 -- start start 2023-11-24T06:28:26.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.646+0000 7ffb87cc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013df80 0x7ffb80152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.646+0000 7ffb87cc9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8013e360 0x7ffb801540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.646+0000 7ffb87cc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb87cc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800b2490 con 0x7ffb8015f9c0 2023-11-24T06:28:26.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb87cc9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb80154ec0 con 0x7ffb8013e360 2023-11-24T06:28:26.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb87cc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb80154be0 con 0x7ffb8013df80 2023-11-24T06:28:26.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43562/0 (socket says 172.21.15.64:43562) 2023-11-24T06:28:26.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 -- 172.21.15.64:0/101807187 learned_addr learned my addr 172.21.15.64:0/101807187 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:26.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb85a65640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013df80 0x7ffb80152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb85264640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8013e360 0x7ffb801540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 -- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013df80 msgr2=0x7ffb80152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013df80 0x7ffb80152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 -- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8013e360 msgr2=0x7ffb801540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8013e360 0x7ffb801540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.647+0000 7ffb86266640 1 -- 172.21.15.64:0/101807187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780094d0 con 0x7ffb8015f9c0 2023-11-24T06:28:26.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.648+0000 7ffb86266640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800afcd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffb780090b0 tx=0x7ffb780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.648+0000 7ffb767fc640 1 -- 172.21.15.64:0/101807187 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7803ac60 con 0x7ffb8015f9c0 2023-11-24T06:28:26.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.648+0000 7ffb767fc640 1 -- 172.21.15.64:0/101807187 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7803adc0 con 0x7ffb8015f9c0 2023-11-24T06:28:26.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.648+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb800b1e40 con 0x7ffb8015f9c0 2023-11-24T06:28:26.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.648+0000 7ffb767fc640 1 -- 172.21.15.64:0/101807187 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78044630 con 0x7ffb8015f9c0 2023-11-24T06:28:26.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.648+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb80153450 con 0x7ffb8015f9c0 2023-11-24T06:28:26.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.649+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb80140890 con 0x7ffb8015f9c0 2023-11-24T06:28:26.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.650+0000 7ffb767fc640 1 -- 172.21.15.64:0/101807187 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb7804d450 con 0x7ffb8015f9c0 2023-11-24T06:28:26.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.651+0000 7ffb767fc640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffb6406ae40 0x7ffb6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.651+0000 7ffb767fc640 1 -- 172.21.15.64:0/101807187 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb780abd20 con 0x7ffb8015f9c0 2023-11-24T06:28:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.651+0000 7ffb85a65640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffb6406ae40 0x7ffb6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.652+0000 7ffb85a65640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffb6406ae40 0x7ffb6406d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0099c0 tx=0x7ffb7c003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.655+0000 7ffb767fc640 1 -- 172.21.15.64:0/101807187 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb78076d30 con 0x7ffb8015f9c0 2023-11-24T06:28:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.851+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb80067000 con 0x7ffb8015f9c0 2023-11-24T06:28:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.852+0000 7ffb767fc640 1 -- 172.21.15.64:0/101807187 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb78035070 con 0x7ffb8015f9c0 2023-11-24T06:28:26.854 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffb6406ae40 msgr2=0x7ffb6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffb6406ae40 0x7ffb6406d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0099c0 tx=0x7ffb7c003990 comp rx=0 tx=0).stop 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 msgr2=0x7ffb800afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800afcd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffb780090b0 tx=0x7ffb780046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 shutdown_connections 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffb6406ae40 0x7ffb6406d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8015f9c0 0x7ffb800afcd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8013e360 0x7ffb801540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 --2- 172.21.15.64:0/101807187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb8013df80 0x7ffb80152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.854+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 >> 172.21.15.64:0/101807187 conn(0x7ffb8006a800 msgr2=0x7ffb8015e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.855+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 shutdown_connections 2023-11-24T06:28:26.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:26.855+0000 7ffb87cc9640 1 -- 172.21.15.64:0/101807187 wait complete. 2023-11-24T06:28:26.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2023-11-24T06:28:27.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.094+0000 7fddc7090640 1 Processor -- start 2023-11-24T06:28:27.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.094+0000 7fddc7090640 1 -- start start 2023-11-24T06:28:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc7090640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fddc013e360 0x7fddc00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc7090640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fddc015f9c0 0x7fddc00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc7090640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc015c5f0 con 0x7fddc013df80 2023-11-24T06:28:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc7090640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc006c470 con 0x7fddc015f9c0 2023-11-24T06:28:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc7090640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc006c750 con 0x7fddc013e360 2023-11-24T06:28:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc562d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fddc015f9c0 0x7fddc00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc562d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fddc015f9c0 0x7fddc00cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43576/0 (socket says 172.21.15.64:43576) 2023-11-24T06:28:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.096+0000 7fddc562d640 1 -- 172.21.15.64:0/406646624 learned_addr learned my addr 172.21.15.64:0/406646624 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.097+0000 7fddb7fff640 1 --2- 172.21.15.64:0/406646624 >> v2:172.21.15.162:3300/0 conn(0x7fddc013e360 0x7fddc00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.097+0000 7fddc562d640 1 -- 172.21.15.64:0/406646624 >> v2:172.21.15.162:3300/0 conn(0x7fddc013e360 msgr2=0x7fddc00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:27.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.097+0000 7fddc562d640 1 --2- 172.21.15.64:0/406646624 >> v2:172.21.15.162:3300/0 conn(0x7fddc013e360 0x7fddc00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.097+0000 7fddc562d640 1 -- 172.21.15.64:0/406646624 >> v1:172.21.15.162:6789/0 conn(0x7fddc013df80 legacy=0x7fddc015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:27.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.097+0000 7fddc562d640 1 -- 172.21.15.64:0/406646624 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc006ca30 con 0x7fddc015f9c0 2023-11-24T06:28:27.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.098+0000 7fddc562d640 1 --2- 172.21.15.64:0/406646624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc015f9c0 0x7fddc00cf7d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fddc00ccae0 tx=0x7fddb802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71284b9dd45971fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:27.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.098+0000 7fddb77fe640 1 -- 172.21.15.64:0/406646624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb802ca50 con 0x7fddc015f9c0 2023-11-24T06:28:27.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.098+0000 7fddb77fe640 1 -- 172.21.15.64:0/406646624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb802cbb0 con 0x7fddc015f9c0 2023-11-24T06:28:27.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.098+0000 7fddb77fe640 1 -- 172.21.15.64:0/406646624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80343f0 con 0x7fddc015f9c0 2023-11-24T06:28:27.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 -- 172.21.15.64:0/406646624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc015f9c0 msgr2=0x7fddc00cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 --2- 172.21.15.64:0/406646624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc015f9c0 0x7fddc00cf7d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fddc00ccae0 tx=0x7fddb802beb0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 -- 172.21.15.64:0/406646624 shutdown_connections 2023-11-24T06:28:27.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 --2- 172.21.15.64:0/406646624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc015f9c0 0x7fddc00cf7d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 --2- 172.21.15.64:0/406646624 >> v2:172.21.15.162:3300/0 conn(0x7fddc013e360 0x7fddc00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 -- 172.21.15.64:0/406646624 >> 172.21.15.64:0/406646624 conn(0x7fddc006a800 msgr2=0x7fddc006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:27.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 -- 172.21.15.64:0/406646624 shutdown_connections 2023-11-24T06:28:27.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.099+0000 7fddc7090640 1 -- 172.21.15.64:0/406646624 wait complete. 2023-11-24T06:28:27.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.101+0000 7fddc7090640 1 Processor -- start 2023-11-24T06:28:27.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.101+0000 7fddc7090640 1 -- start start 2023-11-24T06:28:27.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.102+0000 7fddc7090640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddc013df80 0x7fddc013d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.102+0000 7fddc7090640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc013e360 0x7fddc00b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.102+0000 7fddc7090640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddc015f9c0 0x7fddc00b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.102+0000 7fddc7090640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00aaaa0 con 0x7fddc013e360 2023-11-24T06:28:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.102+0000 7fddc7090640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00a9a40 con 0x7fddc013df80 2023-11-24T06:28:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.102+0000 7fddc7090640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00a9ce0 con 0x7fddc015f9c0 2023-11-24T06:28:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc013e360 0x7fddc00b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc013e360 0x7fddc00b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43584/0 (socket says 172.21.15.64:43584) 2023-11-24T06:28:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 -- 172.21.15.64:0/3846120269 learned_addr learned my addr 172.21.15.64:0/3846120269 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddc4e2c640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddc013df80 0x7fddc013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddc562d640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddc015f9c0 0x7fddc00b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 -- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddc015f9c0 msgr2=0x7fddc00b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddc015f9c0 0x7fddc00b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 -- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddc013df80 msgr2=0x7fddc013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddc013df80 0x7fddc013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 -- 172.21.15.64:0/3846120269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009b30 con 0x7fddc013e360 2023-11-24T06:28:27.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddc4e2c640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddc013df80 0x7fddc013d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:27.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.103+0000 7fddb7fff640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc013e360 0x7fddc00b2cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d7d0 tx=0x7fddb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:27.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.104+0000 7fddb57fa640 1 -- 172.21.15.64:0/3846120269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0018070 con 0x7fddc013e360 2023-11-24T06:28:27.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.104+0000 7fddb57fa640 1 -- 172.21.15.64:0/3846120269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0004020 con 0x7fddc013e360 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.104+0000 7fddb57fa640 1 -- 172.21.15.64:0/3846120269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00041b0 con 0x7fddc013e360 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.104+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc00a9fc0 con 0x7fddc013e360 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.104+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc00aa580 con 0x7fddc013e360 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.105+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc00aa740 con 0x7fddc013e360 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.106+0000 7fddb57fa640 1 -- 172.21.15.64:0/3846120269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fddb0004310 con 0x7fddc013e360 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.107+0000 7fddb57fa640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fddac06ae40 0x7fddac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.107+0000 7fddb57fa640 1 -- 172.21.15.64:0/3846120269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb0089480 con 0x7fddc013e360 2023-11-24T06:28:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.107+0000 7fddc4e2c640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fddac06ae40 0x7fddac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.108+0000 7fddc4e2c640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fddac06ae40 0x7fddac06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fddc006dc30 tx=0x7fddbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.111+0000 7fddb57fa640 1 -- 172.21.15.64:0/3846120269 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fddb0054530 con 0x7fddc013e360 2023-11-24T06:28:27.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.382+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7fddc00675d0 con 0x7fddc013e360 2023-11-24T06:28:28.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:27.997+0000 7fddb57fa640 1 -- 172.21.15.64:0/3846120269 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7fddb0057b50 con 0x7fddc013e360 2023-11-24T06:28:28.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fddac06ae40 msgr2=0x7fddac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fddac06ae40 0x7fddac06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fddc006dc30 tx=0x7fddbc008040 comp rx=0 tx=0).stop 2023-11-24T06:28:28.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc013e360 msgr2=0x7fddc00b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc013e360 0x7fddc00b2cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d7d0 tx=0x7fddb000dc90 comp rx=0 tx=0).stop 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 shutdown_connections 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fddac06ae40 0x7fddac06d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddc015f9c0 0x7fddc00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc013e360 0x7fddc00b2cf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.001+0000 7fddc7090640 1 --2- 172.21.15.64:0/3846120269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddc013df80 0x7fddc013d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.002+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 >> 172.21.15.64:0/3846120269 conn(0x7fddc006a800 msgr2=0x7fddc015a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.002+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 shutdown_connections 2023-11-24T06:28:28.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.002+0000 7fddc7090640 1 -- 172.21.15.64:0/3846120269 wait complete. 2023-11-24T06:28:28.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:28:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.245+0000 7fc2669ef640 1 Processor -- start 2023-11-24T06:28:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.246+0000 7fc2669ef640 1 -- start start 2023-11-24T06:28:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.247+0000 7fc2669ef640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2601447f0 0x7fc26015c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.247+0000 7fc2669ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 0x7fc26015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.247+0000 7fc2669ef640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26015c5c0 con 0x7fc26006c470 2023-11-24T06:28:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.247+0000 7fc2669ef640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc26006d690 con 0x7fc2601447f0 2023-11-24T06:28:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.247+0000 7fc2669ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600cebb0 con 0x7fc2600cccf0 2023-11-24T06:28:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.248+0000 7fc2651ec640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc26006c470 0x7fc2600cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34268/0 (socket says 172.21.15.64:34268) 2023-11-24T06:28:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.248+0000 7fc2651ec640 1 -- 172.21.15.64:0/1569869784 learned_addr learned my addr 172.21.15.64:0/1569869784 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.248+0000 7fc2659ed640 1 --2- 172.21.15.64:0/1569869784 >> v2:172.21.15.64:3300/0 conn(0x7fc2601447f0 0x7fc26015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.248+0000 7fc2661ee640 1 --2- 172.21.15.64:0/1569869784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 0x7fc26015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.248+0000 7fc2649eb640 1 -- 172.21.15.64:0/1569869784 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733561457 0 0) 0x7fc26015c5c0 con 0x7fc26006c470 2023-11-24T06:28:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.249+0000 7fc2649eb640 1 -- 172.21.15.64:0/1569869784 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc24c003520 con 0x7fc26006c470 2023-11-24T06:28:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.249+0000 7fc2659ed640 1 -- 172.21.15.64:0/1569869784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 msgr2=0x7fc26015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.249+0000 7fc2659ed640 1 --2- 172.21.15.64:0/1569869784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 0x7fc26015c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.249+0000 7fc2659ed640 1 -- 172.21.15.64:0/1569869784 >> v1:172.21.15.64:6789/0 conn(0x7fc26006c470 legacy=0x7fc2600cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.249+0000 7fc2659ed640 1 -- 172.21.15.64:0/1569869784 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26006ca10 con 0x7fc2601447f0 2023-11-24T06:28:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.251+0000 7fc2659ed640 1 --2- 172.21.15.64:0/1569869784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc26015c080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc2600cecf0 tx=0x7fc25c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e15df5d23dd768 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.251+0000 7fc2649eb640 1 -- 172.21.15.64:0/1569869784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c031770 con 0x7fc2601447f0 2023-11-24T06:28:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.251+0000 7fc2649eb640 1 -- 172.21.15.64:0/1569869784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c0318d0 con 0x7fc2601447f0 2023-11-24T06:28:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.251+0000 7fc2649eb640 1 -- 172.21.15.64:0/1569869784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c031c60 con 0x7fc2601447f0 2023-11-24T06:28:28.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 -- 172.21.15.64:0/1569869784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 msgr2=0x7fc26015c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 --2- 172.21.15.64:0/1569869784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc26015c080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc2600cecf0 tx=0x7fc25c02b810 comp rx=0 tx=0).stop 2023-11-24T06:28:28.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 -- 172.21.15.64:0/1569869784 shutdown_connections 2023-11-24T06:28:28.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 --2- 172.21.15.64:0/1569869784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 0x7fc26015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 --2- 172.21.15.64:0/1569869784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc26015c080 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 -- 172.21.15.64:0/1569869784 >> 172.21.15.64:0/1569869784 conn(0x7fc26006a3e0 msgr2=0x7fc26006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 -- 172.21.15.64:0/1569869784 shutdown_connections 2023-11-24T06:28:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.252+0000 7fc2669ef640 1 -- 172.21.15.64:0/1569869784 wait complete. 2023-11-24T06:28:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.254+0000 7fc2669ef640 1 Processor -- start 2023-11-24T06:28:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.254+0000 7fc2669ef640 1 -- start start 2023-11-24T06:28:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2669ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc26006c470 0x7fc2601f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2669ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 0x7fc2600aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2669ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc260070530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2669ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc260159f20 con 0x7fc2601447f0 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2669ef640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2601f2b00 con 0x7fc26006c470 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2669ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600af3b0 con 0x7fc2600cccf0 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2661ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc260070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2661ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc260070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43604/0 (socket says 172.21.15.64:43604) 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.255+0000 7fc2661ee640 1 -- 172.21.15.64:0/2551128747 learned_addr learned my addr 172.21.15.64:0/2551128747 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2661ee640 1 -- 172.21.15.64:0/2551128747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 msgr2=0x7fc2600aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2659ed640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc26006c470 0x7fc2601f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:28.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2661ee640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 0x7fc2600aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2661ee640 1 -- 172.21.15.64:0/2551128747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc26006c470 msgr2=0x7fc2601f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2661ee640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc26006c470 0x7fc2601f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2661ee640 1 -- 172.21.15.64:0/2551128747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c009400 con 0x7fc2601447f0 2023-11-24T06:28:28.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2661ee640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc260070530 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc25800efc0 tx=0x7fc25800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:28.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2527fc640 1 -- 172.21.15.64:0/2551128747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258014480 con 0x7fc2601447f0 2023-11-24T06:28:28.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2527fc640 1 -- 172.21.15.64:0/2551128747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25800f040 con 0x7fc2601447f0 2023-11-24T06:28:28.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2600af690 con 0x7fc2601447f0 2023-11-24T06:28:28.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.256+0000 7fc2527fc640 1 -- 172.21.15.64:0/2551128747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258014820 con 0x7fc2601447f0 2023-11-24T06:28:28.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.257+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2600b0390 con 0x7fc2601447f0 2023-11-24T06:28:28.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.258+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2600b1d10 con 0x7fc2601447f0 2023-11-24T06:28:28.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.259+0000 7fc2527fc640 1 -- 172.21.15.64:0/2551128747 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc258014980 con 0x7fc2601447f0 2023-11-24T06:28:28.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.260+0000 7fc2527fc640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc24406ae40 0x7fc24406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:28.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.260+0000 7fc2527fc640 1 -- 172.21.15.64:0/2551128747 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fc25808a510 con 0x7fc2601447f0 2023-11-24T06:28:28.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.260+0000 7fc2659ed640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc24406ae40 0x7fc24406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:28.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.261+0000 7fc2659ed640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc24406ae40 0x7fc24406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc25c036000 tx=0x7fc25c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:28.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.264+0000 7fc2527fc640 1 -- 172.21.15.64:0/2551128747 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2600b1d10 con 0x7fc2601447f0 2023-11-24T06:28:28.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.457+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc2600b1d10 con 0x7fc2601447f0 2023-11-24T06:28:28.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.464+0000 7fc2527fc640 1 -- 172.21.15.64:0/2551128747 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2600b1d10 con 0x7fc2601447f0 2023-11-24T06:28:28.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.466+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc24406ae40 msgr2=0x7fc24406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc24406ae40 0x7fc24406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc25c036000 tx=0x7fc25c03d040 comp rx=0 tx=0).stop 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 msgr2=0x7fc260070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc260070530 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc25800efc0 tx=0x7fc25800c490 comp rx=0 tx=0).stop 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 shutdown_connections 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc24406ae40 0x7fc24406d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2601447f0 0x7fc260070530 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2600cccf0 0x7fc2600aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 --2- 172.21.15.64:0/2551128747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc26006c470 0x7fc2601f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 >> 172.21.15.64:0/2551128747 conn(0x7fc26006a3e0 msgr2=0x7fc2601e7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 shutdown_connections 2023-11-24T06:28:28.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:28.467+0000 7fc2669ef640 1 -- 172.21.15.64:0/2551128747 wait complete. 2023-11-24T06:28:28.488 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:28:28.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:28:28.610 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:28:28.739 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:28:28.739 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:28:28.739 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:28.805 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:28:28.805 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:28.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:29.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.074+0000 7efcde083640 1 Processor -- start 2023-11-24T06:28:29.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.075+0000 7efcde083640 1 -- start start 2023-11-24T06:28:29.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcde083640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcd8150760 0x7efcd8072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcde083640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd80b3710 con 0x7efcd80b32e0 2023-11-24T06:28:29.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcde083640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd8071190 con 0x7efcd815d300 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcde083640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd8071470 con 0x7efcd8150760 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcd77fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcd8150760 0x7efcd8072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcd7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efcd80b32e0 0x7efcd80ce040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34282/0 (socket says 172.21.15.64:34282) 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcd7fff640 1 -- 172.21.15.64:0/1511903907 learned_addr learned my addr 172.21.15.64:0/1511903907 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.076+0000 7efcd77fe640 1 -- 172.21.15.64:0/1511903907 >> v1:172.21.15.162:6789/0 conn(0x7efcd815d300 legacy=0x7efcd80b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.077+0000 7efcd77fe640 1 -- 172.21.15.64:0/1511903907 >> v1:172.21.15.64:6789/0 conn(0x7efcd80b32e0 legacy=0x7efcd80ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.077+0000 7efcd77fe640 1 -- 172.21.15.64:0/1511903907 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8071750 con 0x7efcd8150760 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.077+0000 7efcd6ffd640 1 -- 172.21.15.64:0/1511903907 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129361775 0 0) 0x7efcd80b3710 con 0x7efcd80b32e0 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.078+0000 7efcd77fe640 1 --2- 172.21.15.64:0/1511903907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd8150760 0x7efcd8072230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efcc80097a0 tx=0x7efcc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201cecfc95a25aba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.078+0000 7efcd6ffd640 1 -- 172.21.15.64:0/1511903907 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8031830 con 0x7efcd8150760 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.078+0000 7efcd6ffd640 1 -- 172.21.15.64:0/1511903907 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc8031990 con 0x7efcd8150760 2023-11-24T06:28:29.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.078+0000 7efcd6ffd640 1 -- 172.21.15.64:0/1511903907 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8031d20 con 0x7efcd8150760 2023-11-24T06:28:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.078+0000 7efcde083640 1 -- 172.21.15.64:0/1511903907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd8150760 msgr2=0x7efcd8072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.078+0000 7efcde083640 1 --2- 172.21.15.64:0/1511903907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd8150760 0x7efcd8072230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efcc80097a0 tx=0x7efcc802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.079+0000 7efcde083640 1 -- 172.21.15.64:0/1511903907 shutdown_connections 2023-11-24T06:28:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.079+0000 7efcde083640 1 --2- 172.21.15.64:0/1511903907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd8150760 0x7efcd8072230 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.079+0000 7efcde083640 1 -- 172.21.15.64:0/1511903907 >> 172.21.15.64:0/1511903907 conn(0x7efcd806a800 msgr2=0x7efcd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.079+0000 7efcde083640 1 -- 172.21.15.64:0/1511903907 shutdown_connections 2023-11-24T06:28:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.079+0000 7efcde083640 1 -- 172.21.15.64:0/1511903907 wait complete. 2023-11-24T06:28:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.080+0000 7efcde083640 1 Processor -- start 2023-11-24T06:28:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.080+0000 7efcde083640 1 -- start start 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcde083640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd80b32e0 0x7efcd813cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcde083640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd8150760 0x7efcd8141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcde083640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcd815d300 0x7efcd81454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcde083640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd813d0b0 con 0x7efcd80b32e0 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcde083640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd81389f0 con 0x7efcd8150760 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcde083640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd8145030 con 0x7efcd815d300 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcd77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd80b32e0 0x7efcd813cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcd77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd80b32e0 0x7efcd813cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43622/0 (socket says 172.21.15.64:43622) 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.081+0000 7efcd77fe640 1 -- 172.21.15.64:0/3242375153 learned_addr learned my addr 172.21.15.64:0/3242375153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd77fe640 1 -- 172.21.15.64:0/3242375153 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcd815d300 msgr2=0x7efcd81454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd77fe640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcd815d300 0x7efcd81454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd77fe640 1 -- 172.21.15.64:0/3242375153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd8150760 msgr2=0x7efcd8141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd77fe640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd8150760 0x7efcd8141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd77fe640 1 -- 172.21.15.64:0/3242375153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8009400 con 0x7efcd80b32e0 2023-11-24T06:28:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd77fe640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd80b32e0 0x7efcd813cb70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efcc802bee0 tx=0x7efcc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd4ff9640 1 -- 172.21.15.64:0/3242375153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc803ba70 con 0x7efcd80b32e0 2023-11-24T06:28:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd4ff9640 1 -- 172.21.15.64:0/3242375153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc803bbf0 con 0x7efcd80b32e0 2023-11-24T06:28:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcd4ff9640 1 -- 172.21.15.64:0/3242375153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc80433f0 con 0x7efcd80b32e0 2023-11-24T06:28:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.082+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd8144420 con 0x7efcd80b32e0 2023-11-24T06:28:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.083+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd81449e0 con 0x7efcd80b32e0 2023-11-24T06:28:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.084+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd80675d0 con 0x7efcd80b32e0 2023-11-24T06:28:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.084+0000 7efcd4ff9640 1 -- 172.21.15.64:0/3242375153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcc804c3b0 con 0x7efcd80b32e0 2023-11-24T06:28:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.085+0000 7efcd4ff9640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efcb806ae40 0x7efcb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.085+0000 7efcd4ff9640 1 -- 172.21.15.64:0/3242375153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7efcc80ab630 con 0x7efcd80b32e0 2023-11-24T06:28:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.086+0000 7efccffff640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efcb806ae40 0x7efcb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.086+0000 7efccffff640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efcb806ae40 0x7efcb806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efcd81595c0 tx=0x7efcc0006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:29.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.090+0000 7efcd4ff9640 1 -- 172.21.15.64:0/3242375153 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efcc8076830 con 0x7efcd80b32e0 2023-11-24T06:28:29.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.283+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcd80b1f20 con 0x7efcd80b32e0 2023-11-24T06:28:29.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.286+0000 7efcd4ff9640 1 -- 172.21.15.64:0/3242375153 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7efcd80b1f20 con 0x7efcd80b32e0 2023-11-24T06:28:29.289 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:29.289 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efcb806ae40 msgr2=0x7efcb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efcb806ae40 0x7efcb806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efcd81595c0 tx=0x7efcc0006ba0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd80b32e0 msgr2=0x7efcd813cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd80b32e0 0x7efcd813cb70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efcc802bee0 tx=0x7efcc80046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 shutdown_connections 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efcb806ae40 0x7efcb806d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcd815d300 0x7efcd81454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd8150760 0x7efcd8141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 --2- 172.21.15.64:0/3242375153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd80b32e0 0x7efcd813cb70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 >> 172.21.15.64:0/3242375153 conn(0x7efcd806a800 msgr2=0x7efcd8146a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 shutdown_connections 2023-11-24T06:28:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.289+0000 7efcde083640 1 -- 172.21.15.64:0/3242375153 wait complete. 2023-11-24T06:28:29.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:29.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:29.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.575+0000 7f334ecad640 1 Processor -- start 2023-11-24T06:28:29.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.576+0000 7f334ecad640 1 -- start start 2023-11-24T06:28:29.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.577+0000 7f334ecad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f33480cf9a0 0x7f33480cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.577+0000 7f334ecad640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33480d0320 con 0x7f334815d4b0 2023-11-24T06:28:29.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.577+0000 7f334ecad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33480d1c60 con 0x7f33480b31d0 2023-11-24T06:28:29.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.577+0000 7f334ecad640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480b2730 con 0x7f33480cf9a0 2023-11-24T06:28:29.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.578+0000 7f334dcab640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f334815d4b0 0x7f33480d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34292/0 (socket says 172.21.15.64:34292) 2023-11-24T06:28:29.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.578+0000 7f334dcab640 1 -- 172.21.15.64:0/3510184735 learned_addr learned my addr 172.21.15.64:0/3510184735 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:29.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.578+0000 7f334d4aa640 1 --2- 172.21.15.64:0/3510184735 >> v2:172.21.15.162:3300/0 conn(0x7f33480cf9a0 0x7f33480cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.578+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181888709 0 0) 0x7f33480d0320 con 0x7f334815d4b0 2023-11-24T06:28:29.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.578+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3334003520 con 0x7f334815d4b0 2023-11-24T06:28:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 503546191 0 0) 0x7f3334003520 con 0x7f334815d4b0 2023-11-24T06:28:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 >> v2:172.21.15.162:3300/0 conn(0x7f33480cf9a0 msgr2=0x7f33480cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 --2- 172.21.15.64:0/3510184735 >> v2:172.21.15.162:3300/0 conn(0x7f33480cf9a0 0x7f33480cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 >> v1:172.21.15.162:6789/0 conn(0x7f33480b31d0 legacy=0x7f33480b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480d1ec0 con 0x7f334815d4b0 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f33440019f0 con 0x7f334815d4b0 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 >> v1:172.21.15.64:6789/0 conn(0x7f334815d4b0 legacy=0x7f33480d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33340039c0 0x7f3334003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33340047c0 0x7f3334004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3334005180 0x7f333400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33480b2730 con 0x7f3334005180 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333400c080 con 0x7f33340047c0 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333400c240 con 0x7f33340039c0 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.579+0000 7f334e4ac640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3334005180 0x7f333400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.580+0000 7f334dcab640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33340039c0 0x7f3334003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.580+0000 7f334d4aa640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33340047c0 0x7f3334004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.580+0000 7f334e4ac640 1 -- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33340039c0 msgr2=0x7f3334003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.580+0000 7f334e4ac640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33340039c0 0x7f3334003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.580+0000 7f334e4ac640 1 -- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33340047c0 msgr2=0x7f3334004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.580+0000 7f334e4ac640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33340047c0 0x7f3334004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.580+0000 7f334e4ac640 1 -- 172.21.15.64:0/3510184735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334003520 con 0x7f3334005180 2023-11-24T06:28:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.582+0000 7f334e4ac640 1 -- 172.21.15.64:0/3510184735 reap_dead start 2023-11-24T06:28:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.582+0000 7f334e4ac640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3334005180 0x7f333400b980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3340001730 tx=0x7f33340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf69a24c629dc08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.583+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400052d0 con 0x7f3334005180 2023-11-24T06:28:29.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.583+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334002e050 con 0x7f3334005180 2023-11-24T06:28:29.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.583+0000 7f334cca9640 1 -- 172.21.15.64:0/3510184735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400035f0 con 0x7f3334005180 2023-11-24T06:28:29.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.584+0000 7f334ecad640 1 -- 172.21.15.64:0/3510184735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3334005180 msgr2=0x7f333400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.584+0000 7f334ecad640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3334005180 0x7f333400b980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3340001730 tx=0x7f33340057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.584+0000 7f334ecad640 1 -- 172.21.15.64:0/3510184735 shutdown_connections 2023-11-24T06:28:29.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.584+0000 7f334ecad640 1 --2- 172.21.15.64:0/3510184735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3334005180 0x7f333400b980 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.584+0000 7f334ecad640 1 -- 172.21.15.64:0/3510184735 >> 172.21.15.64:0/3510184735 conn(0x7f33480d3000 msgr2=0x7f33480d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:29.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.584+0000 7f334ecad640 1 -- 172.21.15.64:0/3510184735 shutdown_connections 2023-11-24T06:28:29.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.584+0000 7f334ecad640 1 -- 172.21.15.64:0/3510184735 wait complete. 2023-11-24T06:28:29.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.586+0000 7f334ecad640 1 Processor -- start 2023-11-24T06:28:29.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.586+0000 7f334ecad640 1 -- start start 2023-11-24T06:28:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.587+0000 7f334ecad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3334005180 0x7f33480b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.587+0000 7f334ecad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334809cce0 0x7f33481f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.587+0000 7f334ecad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33480b8640 0x7f33480b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334ecad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3348154cd0 con 0x7f33480b8640 2023-11-24T06:28:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334ecad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33481f6650 con 0x7f334809cce0 2023-11-24T06:28:29.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334ecad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33480b7910 con 0x7f3334005180 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334e4ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33480b8640 0x7f33480b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334e4ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33480b8640 0x7f33480b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43636/0 (socket says 172.21.15.64:43636) 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334e4ac640 1 -- 172.21.15.64:0/3044605667 learned_addr learned my addr 172.21.15.64:0/3044605667 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334dcab640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3334005180 0x7f33480b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.588+0000 7f334d4aa640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334809cce0 0x7f33481f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334e4ac640 1 -- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3334005180 msgr2=0x7f33480b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334e4ac640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3334005180 0x7f33480b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334e4ac640 1 -- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334809cce0 msgr2=0x7f33481f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334e4ac640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334809cce0 0x7f33481f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334e4ac640 1 -- 172.21.15.64:0/3044605667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334809d160 con 0x7f33480b8640 2023-11-24T06:28:29.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334dcab640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3334005180 0x7f33480b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:29.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334d4aa640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334809cce0 0x7f33481f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:29.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f334e4ac640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33480b8640 0x7f33480b8ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3340005040 tx=0x7f3340001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:29.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f333e7fc640 1 -- 172.21.15.64:0/3044605667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340007a30 con 0x7f33480b8640 2023-11-24T06:28:29.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.589+0000 7f333e7fc640 1 -- 172.21.15.64:0/3044605667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334003c420 con 0x7f33480b8640 2023-11-24T06:28:29.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.590+0000 7f333e7fc640 1 -- 172.21.15.64:0/3044605667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334003c7b0 con 0x7f33480b8640 2023-11-24T06:28:29.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.590+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33480b75a0 con 0x7f33480b8640 2023-11-24T06:28:29.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.590+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33480b7e40 con 0x7f33480b8640 2023-11-24T06:28:29.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.591+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3348067450 con 0x7f33480b8640 2023-11-24T06:28:29.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.592+0000 7f333e7fc640 1 -- 172.21.15.64:0/3044605667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f334003a070 con 0x7f33480b8640 2023-11-24T06:28:29.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.594+0000 7f333e7fc640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f332c06ae90 0x7f332c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:29.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.594+0000 7f333e7fc640 1 -- 172.21.15.64:0/3044605667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f3340037070 con 0x7f33480b8640 2023-11-24T06:28:29.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.595+0000 7f334dcab640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f332c06ae90 0x7f332c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:29.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.595+0000 7f334dcab640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f332c06ae90 0x7f332c06d350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3344002230 tx=0x7f3344009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:29.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.599+0000 7f333e7fc640 1 -- 172.21.15.64:0/3044605667 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3340075240 con 0x7f33480b8640 2023-11-24T06:28:29.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.983+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33480cd430 con 0x7f33480b8640 2023-11-24T06:28:29.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.984+0000 7f333e7fc640 1 -- 172.21.15.64:0/3044605667 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3340042020 con 0x7f33480b8640 2023-11-24T06:28:29.988 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f332c06ae90 msgr2=0x7f332c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f332c06ae90 0x7f332c06d350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3344002230 tx=0x7f3344009040 comp rx=0 tx=0).stop 2023-11-24T06:28:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33480b8640 msgr2=0x7f33480b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33480b8640 0x7f33480b8ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3340005040 tx=0x7f3340001120 comp rx=0 tx=0).stop 2023-11-24T06:28:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 shutdown_connections 2023-11-24T06:28:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f332c06ae90 0x7f332c06d350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33480b8640 0x7f33480b8ac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334809cce0 0x7f33481f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 --2- 172.21.15.64:0/3044605667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3334005180 0x7f33480b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 >> 172.21.15.64:0/3044605667 conn(0x7f33480d3000 msgr2=0x7f33480cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.987+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 shutdown_connections 2023-11-24T06:28:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:29.988+0000 7f334ecad640 1 -- 172.21.15.64:0/3044605667 wait complete. 2023-11-24T06:28:30.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:30.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.256+0000 7fe517fff640 1 Processor -- start 2023-11-24T06:28:30.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.256+0000 7fe517fff640 1 -- start start 2023-11-24T06:28:30.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.258+0000 7fe517fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe51815d4b0 0x7fe5180d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.258+0000 7fe517fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe5180b41e0 0x7fe5180cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.258+0000 7fe517fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5180d0320 con 0x7fe5180cf9a0 2023-11-24T06:28:30.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.258+0000 7fe517fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5180b2730 con 0x7fe51815d4b0 2023-11-24T06:28:30.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.258+0000 7fe517fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5180d1ec0 con 0x7fe5180b41e0 2023-11-24T06:28:30.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe5167fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe5180cf9a0 0x7fe5180cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34300/0 (socket says 172.21.15.64:34300) 2023-11-24T06:28:30.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe5167fc640 1 -- 172.21.15.64:0/378179767 learned_addr learned my addr 172.21.15.64:0/378179767 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:30.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe516ffd640 1 --2- 172.21.15.64:0/378179767 >> v2:172.21.15.64:3300/0 conn(0x7fe51815d4b0 0x7fe5180d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe5177fe640 1 --2- 172.21.15.64:0/378179767 >> v2:172.21.15.162:3300/0 conn(0x7fe5180b41e0 0x7fe5180cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe516ffd640 1 -- 172.21.15.64:0/378179767 >> v2:172.21.15.162:3300/0 conn(0x7fe5180b41e0 msgr2=0x7fe5180cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:30.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe516ffd640 1 --2- 172.21.15.64:0/378179767 >> v2:172.21.15.162:3300/0 conn(0x7fe5180b41e0 0x7fe5180cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe516ffd640 1 -- 172.21.15.64:0/378179767 >> v1:172.21.15.64:6789/0 conn(0x7fe5180cf9a0 legacy=0x7fe5180cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.259+0000 7fe516ffd640 1 -- 172.21.15.64:0/378179767 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5180b29d0 con 0x7fe51815d4b0 2023-11-24T06:28:30.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.261+0000 7fe516ffd640 1 --2- 172.21.15.64:0/378179767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51815d4b0 0x7fe5180d1550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe5180d0680 tx=0x7fe50802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11250c8aea23635 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.261+0000 7fe515ffb640 1 -- 172.21.15.64:0/378179767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508031830 con 0x7fe51815d4b0 2023-11-24T06:28:30.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.261+0000 7fe515ffb640 1 -- 172.21.15.64:0/378179767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe508031990 con 0x7fe51815d4b0 2023-11-24T06:28:30.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.261+0000 7fe515ffb640 1 -- 172.21.15.64:0/378179767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508031d20 con 0x7fe51815d4b0 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 -- 172.21.15.64:0/378179767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51815d4b0 msgr2=0x7fe5180d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 --2- 172.21.15.64:0/378179767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51815d4b0 0x7fe5180d1550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe5180d0680 tx=0x7fe50802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 -- 172.21.15.64:0/378179767 shutdown_connections 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 --2- 172.21.15.64:0/378179767 >> v2:172.21.15.162:3300/0 conn(0x7fe5180b41e0 0x7fe5180cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 --2- 172.21.15.64:0/378179767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51815d4b0 0x7fe5180d1550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 -- 172.21.15.64:0/378179767 >> 172.21.15.64:0/378179767 conn(0x7fe5180d3000 msgr2=0x7fe5180d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 -- 172.21.15.64:0/378179767 shutdown_connections 2023-11-24T06:28:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.262+0000 7fe517fff640 1 -- 172.21.15.64:0/378179767 wait complete. 2023-11-24T06:28:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.264+0000 7fe517fff640 1 Processor -- start 2023-11-24T06:28:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.264+0000 7fe517fff640 1 -- start start 2023-11-24T06:28:30.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.265+0000 7fe517fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5180b41e0 0x7fe51809f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.265+0000 7fe517fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5180cf9a0 0x7fe5180b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.265+0000 7fe517fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe51815d4b0 0x7fe5180b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe517fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51809fb50 con 0x7fe5180cf9a0 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe517fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51809fe30 con 0x7fe5180b41e0 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe517fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180a0110 con 0x7fe51815d4b0 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe516ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5180b41e0 0x7fe51809f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe5167fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5180cf9a0 0x7fe5180b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe516ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5180b41e0 0x7fe51809f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:50244/0 (socket says 172.21.15.64:50244) 2023-11-24T06:28:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe516ffd640 1 -- 172.21.15.64:0/1913901202 learned_addr learned my addr 172.21.15.64:0/1913901202 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe5177fe640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe51815d4b0 0x7fe5180b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe516ffd640 1 -- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe51815d4b0 msgr2=0x7fe5180b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe516ffd640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe51815d4b0 0x7fe5180b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe516ffd640 1 -- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5180cf9a0 msgr2=0x7fe5180b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.266+0000 7fe516ffd640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5180cf9a0 0x7fe5180b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.267+0000 7fe516ffd640 1 -- 172.21.15.64:0/1913901202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe508009400 con 0x7fe5180b41e0 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.267+0000 7fe5177fe640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe51815d4b0 0x7fe5180b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.267+0000 7fe516ffd640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5180b41e0 0x7fe51809f1e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe50802b910 tx=0x7fe5080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.267+0000 7fe4fffff640 1 -- 172.21.15.64:0/1913901202 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5080317d0 con 0x7fe5180b41e0 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.267+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51815da10 con 0x7fe5180b41e0 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.267+0000 7fe4fffff640 1 -- 172.21.15.64:0/1913901202 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50803aa60 con 0x7fe5180b41e0 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.267+0000 7fe4fffff640 1 -- 172.21.15.64:0/1913901202 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50803add0 con 0x7fe5180b41e0 2023-11-24T06:28:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.268+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5180a06d0 con 0x7fe5180b41e0 2023-11-24T06:28:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.269+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe518067450 con 0x7fe5180b41e0 2023-11-24T06:28:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.270+0000 7fe4fffff640 1 -- 172.21.15.64:0/1913901202 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe50804d440 con 0x7fe5180b41e0 2023-11-24T06:28:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.271+0000 7fe4fffff640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe50406ae40 0x7fe50406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.271+0000 7fe4fffff640 1 -- 172.21.15.64:0/1913901202 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fe5080abd20 con 0x7fe5180b41e0 2023-11-24T06:28:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.272+0000 7fe5167fc640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe50406ae40 0x7fe50406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.272+0000 7fe5167fc640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe50406ae40 0x7fe50406d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5180cac90 tx=0x7fe50c007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.285+0000 7fe4fffff640 1 -- 172.21.15.64:0/1913901202 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe508076cd0 con 0x7fe5180b41e0 2023-11-24T06:28:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.550+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5180b5860 con 0x7fe5180b41e0 2023-11-24T06:28:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.551+0000 7fe4fffff640 1 -- 172.21.15.64:0/1913901202 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe50807a2f0 con 0x7fe5180b41e0 2023-11-24T06:28:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe50406ae40 msgr2=0x7fe50406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe50406ae40 0x7fe50406d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5180cac90 tx=0x7fe50c007e80 comp rx=0 tx=0).stop 2023-11-24T06:28:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5180b41e0 msgr2=0x7fe51809f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5180b41e0 0x7fe51809f1e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe50802b910 tx=0x7fe5080046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 shutdown_connections 2023-11-24T06:28:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe50406ae40 0x7fe50406d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe51815d4b0 0x7fe5180b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5180cf9a0 0x7fe5180b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 --2- 172.21.15.64:0/1913901202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5180b41e0 0x7fe51809f1e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 >> 172.21.15.64:0/1913901202 conn(0x7fe5180d3000 msgr2=0x7fe5180b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:30.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 shutdown_connections 2023-11-24T06:28:30.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.554+0000 7fe517fff640 1 -- 172.21.15.64:0/1913901202 wait complete. 2023-11-24T06:28:30.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:30.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.770+0000 7f02f542e640 1 Processor -- start 2023-11-24T06:28:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.770+0000 7f02f542e640 1 -- start start 2023-11-24T06:28:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f542e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00cf9a0 0x7f02f00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f542e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f02f00b31d0 0x7f02f00b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f542e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f00d0320 con 0x7f02f015d4b0 2023-11-24T06:28:30.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f542e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f00d1c60 con 0x7f02f00b31d0 2023-11-24T06:28:30.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f542e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f00b2730 con 0x7f02f00cf9a0 2023-11-24T06:28:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f4c2d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f02f00b31d0 0x7f02f00b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02ef7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00cf9a0 0x7f02f00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f4c2d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f02f00b31d0 0x7f02f00b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43668/0 (socket says 172.21.15.64:43668) 2023-11-24T06:28:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.772+0000 7f02f4c2d640 1 -- 172.21.15.64:0/1499201068 learned_addr learned my addr 172.21.15.64:0/1499201068 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.773+0000 7f02f4c2d640 1 -- 172.21.15.64:0/1499201068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00cf9a0 msgr2=0x7f02f00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.773+0000 7f02f4c2d640 1 --2- 172.21.15.64:0/1499201068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00cf9a0 0x7f02f00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.773+0000 7f02f4c2d640 1 -- 172.21.15.64:0/1499201068 >> v1:172.21.15.162:6789/0 conn(0x7f02f015d4b0 legacy=0x7f02f00d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.773+0000 7f02f4c2d640 1 -- 172.21.15.64:0/1499201068 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00d1ec0 con 0x7f02f00b31d0 2023-11-24T06:28:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.774+0000 7f02f4c2d640 1 --2- 172.21.15.64:0/1499201068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00b31d0 0x7f02f00b3660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009e80 tx=0x7f02e802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f40300ca1265588 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.775+0000 7f02eeffd640 1 -- 172.21.15.64:0/1499201068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e802caf0 con 0x7f02f00b31d0 2023-11-24T06:28:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.775+0000 7f02eeffd640 1 -- 172.21.15.64:0/1499201068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e802cc50 con 0x7f02f00b31d0 2023-11-24T06:28:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.775+0000 7f02eeffd640 1 -- 172.21.15.64:0/1499201068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e80343f0 con 0x7f02f00b31d0 2023-11-24T06:28:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.775+0000 7f02f542e640 1 -- 172.21.15.64:0/1499201068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00b31d0 msgr2=0x7f02f00b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.775+0000 7f02f542e640 1 --2- 172.21.15.64:0/1499201068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00b31d0 0x7f02f00b3660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009e80 tx=0x7f02e802bf50 comp rx=0 tx=0).stop 2023-11-24T06:28:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.776+0000 7f02f542e640 1 -- 172.21.15.64:0/1499201068 shutdown_connections 2023-11-24T06:28:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.776+0000 7f02f542e640 1 --2- 172.21.15.64:0/1499201068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00b31d0 0x7f02f00b3660 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.776+0000 7f02f542e640 1 --2- 172.21.15.64:0/1499201068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00cf9a0 0x7f02f00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.776+0000 7f02f542e640 1 -- 172.21.15.64:0/1499201068 >> 172.21.15.64:0/1499201068 conn(0x7f02f00d3000 msgr2=0x7f02f00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:30.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.776+0000 7f02f542e640 1 -- 172.21.15.64:0/1499201068 shutdown_connections 2023-11-24T06:28:30.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.776+0000 7f02f542e640 1 -- 172.21.15.64:0/1499201068 wait complete. 2023-11-24T06:28:30.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.778+0000 7f02f542e640 1 Processor -- start 2023-11-24T06:28:30.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.778+0000 7f02f542e640 1 -- start start 2023-11-24T06:28:30.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02f542e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00b31d0 0x7f02f015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02f542e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00cf9a0 0x7f02f0157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02f542e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02f015d4b0 0x7f02f01f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02f542e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015f5e0 con 0x7f02f00cf9a0 2023-11-24T06:28:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02f542e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f01575d0 con 0x7f02f015d4b0 2023-11-24T06:28:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02f542e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015f300 con 0x7f02f00b31d0 2023-11-24T06:28:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02ef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00cf9a0 0x7f02f0157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02ef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00cf9a0 0x7f02f0157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43676/0 (socket says 172.21.15.64:43676) 2023-11-24T06:28:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02ef7fe640 1 -- 172.21.15.64:0/999062442 learned_addr learned my addr 172.21.15.64:0/999062442 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.779+0000 7f02f4c2d640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02f015d4b0 0x7f02f01f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02effff640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00b31d0 0x7f02f015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ef7fe640 1 -- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00b31d0 msgr2=0x7f02f015fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ef7fe640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00b31d0 0x7f02f015fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ef7fe640 1 -- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02f015d4b0 msgr2=0x7f02f01f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ef7fe640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02f015d4b0 0x7f02f01f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ef7fe640 1 -- 172.21.15.64:0/999062442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e8009b30 con 0x7f02f00cf9a0 2023-11-24T06:28:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02effff640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00b31d0 0x7f02f015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:30.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ef7fe640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00cf9a0 0x7f02f0157090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e870 tx=0x7f02e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ecff9640 1 -- 172.21.15.64:0/999062442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e40094f0 con 0x7f02f00cf9a0 2023-11-24T06:28:30.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.780+0000 7f02ecff9640 1 -- 172.21.15.64:0/999062442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400c940 con 0x7f02f00cf9a0 2023-11-24T06:28:30.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.781+0000 7f02ecff9640 1 -- 172.21.15.64:0/999062442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400ccf0 con 0x7f02f00cf9a0 2023-11-24T06:28:30.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.781+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f00ca170 con 0x7f02f00cf9a0 2023-11-24T06:28:30.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.781+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f01f7110 con 0x7f02f00cf9a0 2023-11-24T06:28:30.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.783+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f00ca970 con 0x7f02f00cf9a0 2023-11-24T06:28:30.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.783+0000 7f02ecff9640 1 -- 172.21.15.64:0/999062442 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02e4013470 con 0x7f02f00cf9a0 2023-11-24T06:28:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.785+0000 7f02ecff9640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f02dc06ae40 0x7f02dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.785+0000 7f02ecff9640 1 -- 172.21.15.64:0/999062442 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f02e40897a0 con 0x7f02f00cf9a0 2023-11-24T06:28:30.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.787+0000 7f02effff640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f02dc06ae40 0x7f02dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.787+0000 7f02effff640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f02dc06ae40 0x7f02dc06d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000f60 tx=0x7f02e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.790+0000 7f02ecff9640 1 -- 172.21.15.64:0/999062442 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02f00ca970 con 0x7f02f00cf9a0 2023-11-24T06:28:30.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.995+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02f00ca970 con 0x7f02f00cf9a0 2023-11-24T06:28:30.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.996+0000 7f02ecff9640 1 -- 172.21.15.64:0/999062442 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02f00ca970 con 0x7f02f00cf9a0 2023-11-24T06:28:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.998+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f02dc06ae40 msgr2=0x7f02dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.998+0000 7f02f542e640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f02dc06ae40 0x7f02dc06d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000f60 tx=0x7f02e0009040 comp rx=0 tx=0).stop 2023-11-24T06:28:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.998+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00cf9a0 msgr2=0x7f02f0157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.998+0000 7f02f542e640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00cf9a0 0x7f02f0157090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e870 tx=0x7f02e400ed30 comp rx=0 tx=0).stop 2023-11-24T06:28:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 shutdown_connections 2023-11-24T06:28:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f02dc06ae40 0x7f02dc06d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02f015d4b0 0x7f02f01f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02f00cf9a0 0x7f02f0157090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 --2- 172.21.15.64:0/999062442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02f00b31d0 0x7f02f015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 >> 172.21.15.64:0/999062442 conn(0x7f02f00d3000 msgr2=0x7f02f006c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 shutdown_connections 2023-11-24T06:28:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:30.999+0000 7f02f542e640 1 -- 172.21.15.64:0/999062442 wait complete. 2023-11-24T06:28:31.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:31.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.249+0000 7f89b2c67640 1 Processor -- start 2023-11-24T06:28:31.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.249+0000 7f89b2c67640 1 -- start start 2023-11-24T06:28:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.250+0000 7f89b2c67640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89ac160920 0x7f89ac0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.250+0000 7f89b2c67640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f89ac0cfea0 0x7f89ac161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.250+0000 7f89b2c67640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ac0d0a10 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.250+0000 7f89b2c67640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac0d1f50 con 0x7f89ac160920 2023-11-24T06:28:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.250+0000 7f89b2c67640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac0cf080 con 0x7f89ac0cfea0 2023-11-24T06:28:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.251+0000 7f89b1c65640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89ac160920 0x7f89ac0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.251+0000 7f89b1c65640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89ac160920 0x7f89ac0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43680/0 (socket says 172.21.15.64:43680) 2023-11-24T06:28:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.251+0000 7f89b1c65640 1 -- 172.21.15.64:0/1940059876 learned_addr learned my addr 172.21.15.64:0/1940059876 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.251+0000 7f89b1c65640 1 -- 172.21.15.64:0/1940059876 >> v2:172.21.15.162:3300/0 conn(0x7f89ac0cfea0 msgr2=0x7f89ac161720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:28:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.251+0000 7f89b1c65640 1 --2- 172.21.15.64:0/1940059876 >> v2:172.21.15.162:3300/0 conn(0x7f89ac0cfea0 0x7f89ac161720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.251+0000 7f89b1c65640 1 -- 172.21.15.64:0/1940059876 >> v1:172.21.15.162:6789/0 conn(0x7f89ac0cf9f0 legacy=0x7f89ac161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.251+0000 7f89b1c65640 1 -- 172.21.15.64:0/1940059876 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0d0310 con 0x7f89ac160920 2023-11-24T06:28:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.252+0000 7f89b1c65640 1 --2- 172.21.15.64:0/1940059876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac160920 0x7f89ac0b3f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89ac08c2c0 tx=0x7f89a802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124ae4f09ccd936f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.253+0000 7f89b0c63640 1 -- 172.21.15.64:0/1940059876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031890 con 0x7f89ac160920 2023-11-24T06:28:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.253+0000 7f89b0c63640 1 -- 172.21.15.64:0/1940059876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a80319f0 con 0x7f89ac160920 2023-11-24T06:28:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.253+0000 7f89b0c63640 1 -- 172.21.15.64:0/1940059876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031d80 con 0x7f89ac160920 2023-11-24T06:28:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 -- 172.21.15.64:0/1940059876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac160920 msgr2=0x7f89ac0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 --2- 172.21.15.64:0/1940059876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac160920 0x7f89ac0b3f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89ac08c2c0 tx=0x7f89a802b930 comp rx=0 tx=0).stop 2023-11-24T06:28:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 -- 172.21.15.64:0/1940059876 shutdown_connections 2023-11-24T06:28:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 --2- 172.21.15.64:0/1940059876 >> v2:172.21.15.162:3300/0 conn(0x7f89ac0cfea0 0x7f89ac161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 --2- 172.21.15.64:0/1940059876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac160920 0x7f89ac0b3f30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 -- 172.21.15.64:0/1940059876 >> 172.21.15.64:0/1940059876 conn(0x7f89ac0d3090 msgr2=0x7f89ac0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 -- 172.21.15.64:0/1940059876 shutdown_connections 2023-11-24T06:28:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.254+0000 7f89b2c67640 1 -- 172.21.15.64:0/1940059876 wait complete. 2023-11-24T06:28:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.257+0000 7f89b2c67640 1 Processor -- start 2023-11-24T06:28:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.257+0000 7f89b2c67640 1 -- start start 2023-11-24T06:28:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.258+0000 7f89b2c67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac0cf9f0 0x7f89ac0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.258+0000 7f89b2c67640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac0cfea0 0x7f89ac0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.258+0000 7f89b2c67640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ac160920 0x7f89ac0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b2c67640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac151810 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b1c65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac0cf9f0 0x7f89ac0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b2c67640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0ca7f0 con 0x7f89ac160920 2023-11-24T06:28:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b2c67640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0caad0 con 0x7f89ac0cfea0 2023-11-24T06:28:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b1c65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac0cf9f0 0x7f89ac0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43696/0 (socket says 172.21.15.64:43696) 2023-11-24T06:28:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b1c65640 1 -- 172.21.15.64:0/748519134 learned_addr learned my addr 172.21.15.64:0/748519134 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:31.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b1464640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac0cfea0 0x7f89ac0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.259+0000 7f89b2466640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ac160920 0x7f89ac0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.260+0000 7f89b1c65640 1 -- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac0cfea0 msgr2=0x7f89ac0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.260+0000 7f89b1c65640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac0cfea0 0x7f89ac0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.260+0000 7f89b1c65640 1 -- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ac160920 msgr2=0x7f89ac0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.260+0000 7f89b1c65640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ac160920 0x7f89ac0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.260+0000 7f89b1c65640 1 -- 172.21.15.64:0/748519134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80094d0 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.260+0000 7f89b1c65640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac0cf9f0 0x7f89ac0ca160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f89a8000c00 tx=0x7f89a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.261+0000 7f89a27fc640 1 -- 172.21.15.64:0/748519134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031800 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.261+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac0cadb0 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.261+0000 7f89a27fc640 1 -- 172.21.15.64:0/748519134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a80443f0 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.261+0000 7f89a27fc640 1 -- 172.21.15.64:0/748519134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8044780 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.261+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac0cb370 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.263+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac067450 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.266+0000 7f89a27fc640 1 -- 172.21.15.64:0/748519134 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89a804d450 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.268+0000 7f89a27fc640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f899006af10 0x7f899006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.268+0000 7f89a27fc640 1 -- 172.21.15.64:0/748519134 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f89a80ac170 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.268+0000 7f89b1464640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f899006af10 0x7f899006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.269+0000 7f89b1464640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f899006af10 0x7f899006d3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0668b0 tx=0x7f899c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.271+0000 7f89a27fc640 1 -- 172.21.15.64:0/748519134 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89a8077250 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.549+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f89ac152060 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.550+0000 7f89a27fc640 1 -- 172.21.15.64:0/748519134 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f89a807a870 con 0x7f89ac0cf9f0 2023-11-24T06:28:31.552 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:31.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f899006af10 msgr2=0x7f899006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f899006af10 0x7f899006d3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0668b0 tx=0x7f899c009060 comp rx=0 tx=0).stop 2023-11-24T06:28:31.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac0cf9f0 msgr2=0x7f89ac0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac0cf9f0 0x7f89ac0ca160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f89a8000c00 tx=0x7f89a80046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 shutdown_connections 2023-11-24T06:28:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f899006af10 0x7f899006d3d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ac160920 0x7f89ac0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac0cfea0 0x7f89ac0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 --2- 172.21.15.64:0/748519134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ac0cf9f0 0x7f89ac0ca160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.553+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 >> 172.21.15.64:0/748519134 conn(0x7f89ac0d3090 msgr2=0x7f89ac06c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.554+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 shutdown_connections 2023-11-24T06:28:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.554+0000 7f89b2c67640 1 -- 172.21.15.64:0/748519134 wait complete. 2023-11-24T06:28:31.580 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:28:31.580 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:31.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:31.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.886+0000 7fe939eff640 1 Processor -- start 2023-11-24T06:28:31.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.886+0000 7fe939eff640 1 -- start start 2023-11-24T06:28:31.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe939eff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9340cfea0 0x7fe9340ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe939eff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9340d1d60 con 0x7fe934160d40 2023-11-24T06:28:31.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe939eff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9340d3d20 con 0x7fe934160960 2023-11-24T06:28:31.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe939eff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9340d4000 con 0x7fe9340cfea0 2023-11-24T06:28:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe9396fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9340cfea0 0x7fe9340ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe9396fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9340cfea0 0x7fe9340ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43712/0 (socket says 172.21.15.64:43712) 2023-11-24T06:28:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe9396fe640 1 -- 172.21.15.64:0/1104072453 learned_addr learned my addr 172.21.15.64:0/1104072453 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.888+0000 7fe9396fe640 1 -- 172.21.15.64:0/1104072453 >> v1:172.21.15.162:6789/0 conn(0x7fe934160960 legacy=0x7fe9340cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.889+0000 7fe9396fe640 1 -- 172.21.15.64:0/1104072453 >> v1:172.21.15.64:6789/0 conn(0x7fe934160d40 legacy=0x7fe9340cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.889+0000 7fe9396fe640 1 -- 172.21.15.64:0/1104072453 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340d19f0 con 0x7fe9340cfea0 2023-11-24T06:28:31.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.890+0000 7fe9396fe640 1 --2- 172.21.15.64:0/1104072453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9340cfea0 0x7fe9340ce380 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009e50 tx=0x7fe92c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6046afe9807500ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.890+0000 7fe9337fe640 1 -- 172.21.15.64:0/1104072453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c02cb90 con 0x7fe9340cfea0 2023-11-24T06:28:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.890+0000 7fe9337fe640 1 -- 172.21.15.64:0/1104072453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c02ccf0 con 0x7fe9340cfea0 2023-11-24T06:28:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.890+0000 7fe9337fe640 1 -- 172.21.15.64:0/1104072453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c034610 con 0x7fe9340cfea0 2023-11-24T06:28:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.891+0000 7fe939eff640 1 -- 172.21.15.64:0/1104072453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9340cfea0 msgr2=0x7fe9340ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.891+0000 7fe939eff640 1 --2- 172.21.15.64:0/1104072453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9340cfea0 0x7fe9340ce380 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009e50 tx=0x7fe92c02bff0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.891+0000 7fe939eff640 1 -- 172.21.15.64:0/1104072453 shutdown_connections 2023-11-24T06:28:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.891+0000 7fe939eff640 1 --2- 172.21.15.64:0/1104072453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9340cfea0 0x7fe9340ce380 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.891+0000 7fe939eff640 1 -- 172.21.15.64:0/1104072453 >> 172.21.15.64:0/1104072453 conn(0x7fe9340d3060 msgr2=0x7fe9340d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.891+0000 7fe939eff640 1 -- 172.21.15.64:0/1104072453 shutdown_connections 2023-11-24T06:28:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.891+0000 7fe939eff640 1 -- 172.21.15.64:0/1104072453 wait complete. 2023-11-24T06:28:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.893+0000 7fe939eff640 1 Processor -- start 2023-11-24T06:28:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.893+0000 7fe939eff640 1 -- start start 2023-11-24T06:28:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.894+0000 7fe939eff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9340cfea0 0x7fe934159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.894+0000 7fe939eff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934160960 0x7fe9340c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.894+0000 7fe939eff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe934160d40 0x7fe934153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.894+0000 7fe939eff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c8650 con 0x7fe934160960 2023-11-24T06:28:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.894+0000 7fe939eff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c8930 con 0x7fe934160d40 2023-11-24T06:28:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.894+0000 7fe939eff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c8c10 con 0x7fe9340cfea0 2023-11-24T06:28:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934160960 0x7fe9340c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934160960 0x7fe9340c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43718/0 (socket says 172.21.15.64:43718) 2023-11-24T06:28:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 -- 172.21.15.64:0/518930506 learned_addr learned my addr 172.21.15.64:0/518930506 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe938efd640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9340cfea0 0x7fe934159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe9396fe640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe934160d40 0x7fe934153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 -- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9340cfea0 msgr2=0x7fe934159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9340cfea0 0x7fe934159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 -- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe934160d40 msgr2=0x7fe934153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe934160d40 0x7fe934153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.895+0000 7fe933fff640 1 -- 172.21.15.64:0/518930506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c009b30 con 0x7fe934160960 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.896+0000 7fe933fff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934160960 0x7fe9340c7d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c3c0 tx=0x7fe92000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.896+0000 7fe9317fa640 1 -- 172.21.15.64:0/518930506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200052d0 con 0x7fe934160960 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.896+0000 7fe9317fa640 1 -- 172.21.15.64:0/518930506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe920005450 con 0x7fe934160960 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.896+0000 7fe9317fa640 1 -- 172.21.15.64:0/518930506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920003830 con 0x7fe934160960 2023-11-24T06:28:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.896+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9340c8ef0 con 0x7fe934160960 2023-11-24T06:28:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.896+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9340c8280 con 0x7fe934160960 2023-11-24T06:28:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.898+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe934067450 con 0x7fe934160960 2023-11-24T06:28:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.900+0000 7fe9317fa640 1 -- 172.21.15.64:0/518930506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe920003990 con 0x7fe934160960 2023-11-24T06:28:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.901+0000 7fe9317fa640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe91806ae70 0x7fe91806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.901+0000 7fe9317fa640 1 -- 172.21.15.64:0/518930506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fe920088be0 con 0x7fe934160960 2023-11-24T06:28:31.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.901+0000 7fe938efd640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe91806ae70 0x7fe91806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.902+0000 7fe938efd640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe91806ae70 0x7fe91806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe928001780 tx=0x7fe928009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:31.908+0000 7fe9317fa640 1 -- 172.21.15.64:0/518930506 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe920053c40 con 0x7fe934160960 2023-11-24T06:28:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.123+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9341e62e0 con 0x7fe934160960 2023-11-24T06:28:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.125+0000 7fe9317fa640 1 -- 172.21.15.64:0/518930506 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7fe920017090 con 0x7fe934160960 2023-11-24T06:28:32.128 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:32.128 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe91806ae70 msgr2=0x7fe91806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe91806ae70 0x7fe91806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe928001780 tx=0x7fe928009040 comp rx=0 tx=0).stop 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934160960 msgr2=0x7fe9340c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934160960 0x7fe9340c7d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c3c0 tx=0x7fe92000c880 comp rx=0 tx=0).stop 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 shutdown_connections 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe91806ae70 0x7fe91806d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe934160d40 0x7fe934153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934160960 0x7fe9340c7d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 --2- 172.21.15.64:0/518930506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9340cfea0 0x7fe934159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 >> 172.21.15.64:0/518930506 conn(0x7fe9340d3060 msgr2=0x7fe9340cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 shutdown_connections 2023-11-24T06:28:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.127+0000 7fe939eff640 1 -- 172.21.15.64:0/518930506 wait complete. 2023-11-24T06:28:32.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:32.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.363+0000 7f8904844640 1 Processor -- start 2023-11-24T06:28:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.363+0000 7f8904844640 1 -- start start 2023-11-24T06:28:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.364+0000 7f8904844640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f890015d960 0x7f89000cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.364+0000 7f8904844640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89000ce760 con 0x7f89000b20d0 2023-11-24T06:28:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.364+0000 7f8904844640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89000cf250 con 0x7f890015d580 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.364+0000 7f8904844640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89000d0a00 con 0x7f890015d960 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.365+0000 7f88fffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f89000b20d0 0x7f89000b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34310/0 (socket says 172.21.15.64:34310) 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.365+0000 7f88fffff640 1 -- 172.21.15.64:0/3712024475 learned_addr learned my addr 172.21.15.64:0/3712024475 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.365+0000 7f88feffd640 1 --2- 172.21.15.64:0/3712024475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f890015d960 0x7f89000cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.365+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698981861 0 0) 0x7f89000ce760 con 0x7f89000b20d0 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.365+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e8003520 con 0x7f89000b20d0 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.366+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141348567 0 0) 0x7f89000cf250 con 0x7f890015d580 2023-11-24T06:28:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.366+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89000ce760 con 0x7f890015d580 2023-11-24T06:28:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.366+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2147415167 0 0) 0x7f88e8003520 con 0x7f89000b20d0 2023-11-24T06:28:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.366+0000 7f88feffd640 1 -- 172.21.15.64:0/3712024475 >> v1:172.21.15.162:6789/0 conn(0x7f890015d580 legacy=0x7f89000cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.366+0000 7f88feffd640 1 -- 172.21.15.64:0/3712024475 >> v1:172.21.15.64:6789/0 conn(0x7f89000b20d0 legacy=0x7f89000b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.366+0000 7f88feffd640 1 -- 172.21.15.64:0/3712024475 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000d1f50 con 0x7f890015d960 2023-11-24T06:28:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.368+0000 7f88feffd640 1 --2- 172.21.15.64:0/3712024475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f890015d960 0x7f89000cfd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097d0 tx=0x7f88f402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6fdb9bbfb66515c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.368+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f80019f0 con 0x7f890015d960 2023-11-24T06:28:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.368+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f402e050 con 0x7f890015d960 2023-11-24T06:28:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.368+0000 7f88fe7fc640 1 -- 172.21.15.64:0/3712024475 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4030710 con 0x7f890015d960 2023-11-24T06:28:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.369+0000 7f8904844640 1 -- 172.21.15.64:0/3712024475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f890015d960 msgr2=0x7f89000cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.369+0000 7f8904844640 1 --2- 172.21.15.64:0/3712024475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f890015d960 0x7f89000cfd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097d0 tx=0x7f88f402b790 comp rx=0 tx=0).stop 2023-11-24T06:28:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.369+0000 7f8904844640 1 -- 172.21.15.64:0/3712024475 shutdown_connections 2023-11-24T06:28:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.369+0000 7f8904844640 1 --2- 172.21.15.64:0/3712024475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f890015d960 0x7f89000cfd60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.369+0000 7f8904844640 1 -- 172.21.15.64:0/3712024475 >> 172.21.15.64:0/3712024475 conn(0x7f89000d3040 msgr2=0x7f89000d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.370+0000 7f8904844640 1 -- 172.21.15.64:0/3712024475 shutdown_connections 2023-11-24T06:28:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.370+0000 7f8904844640 1 -- 172.21.15.64:0/3712024475 wait complete. 2023-11-24T06:28:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.372+0000 7f8904844640 1 Processor -- start 2023-11-24T06:28:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.373+0000 7f8904844640 1 -- start start 2023-11-24T06:28:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.374+0000 7f8904844640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89000b20d0 0x7f89000b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.374+0000 7f8904844640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890015d580 0x7f89000ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.374+0000 7f8904844640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f890015d960 0x7f89000ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.374+0000 7f8904844640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89000b8580 con 0x7f890015d580 2023-11-24T06:28:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.374+0000 7f8904844640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89001f3220 con 0x7f890015d960 2023-11-24T06:28:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.374+0000 7f8904844640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89000b74e0 con 0x7f89000b20d0 2023-11-24T06:28:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890015d580 0x7f89000ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890015d580 0x7f89000ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43728/0 (socket says 172.21.15.64:43728) 2023-11-24T06:28:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89000b20d0 0x7f89000b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 -- 172.21.15.64:0/1052202383 learned_addr learned my addr 172.21.15.64:0/1052202383 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88fffff640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f890015d960 0x7f89000ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 -- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89000b20d0 msgr2=0x7f89000b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89000b20d0 0x7f89000b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 -- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f890015d960 msgr2=0x7f89000ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f890015d960 0x7f89000ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88feffd640 1 -- 172.21.15.64:0/1052202383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4009400 con 0x7f890015d580 2023-11-24T06:28:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.375+0000 7f88fffff640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f890015d960 0x7f89000ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.376+0000 7f88feffd640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890015d580 0x7f89000ca110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f88f40098f0 tx=0x7f88f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.376+0000 7f88e7fff640 1 -- 172.21.15.64:0/1052202383 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4003ea0 con 0x7f890015d580 2023-11-24T06:28:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.376+0000 7f88e7fff640 1 -- 172.21.15.64:0/1052202383 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f4006e10 con 0x7f890015d580 2023-11-24T06:28:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.376+0000 7f88e7fff640 1 -- 172.21.15.64:0/1052202383 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4030750 con 0x7f890015d580 2023-11-24T06:28:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.376+0000 7f8904844640 1 -- 172.21.15.64:0/1052202383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89000b77c0 con 0x7f890015d580 2023-11-24T06:28:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.376+0000 7f8904844640 1 -- 172.21.15.64:0/1052202383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89000b7d80 con 0x7f890015d580 2023-11-24T06:28:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.378+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d8005250 con 0x7f890015d580 2023-11-24T06:28:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.380+0000 7f88e7fff640 1 -- 172.21.15.64:0/1052202383 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88f4039070 con 0x7f890015d580 2023-11-24T06:28:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.381+0000 7f88e7fff640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f88ec06ae90 0x7f88ec06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.381+0000 7f88e7fff640 1 -- 172.21.15.64:0/1052202383 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f88f40aaff0 con 0x7f890015d580 2023-11-24T06:28:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.381+0000 7f88ff7fe640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f88ec06ae90 0x7f88ec06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.382+0000 7f88ff7fe640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f88ec06ae90 0x7f88ec06d350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88f0007fd0 tx=0x7f88f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.385+0000 7f88e7fff640 1 -- 172.21.15.64:0/1052202383 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88f40dde90 con 0x7f890015d580 2023-11-24T06:28:32.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.592+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88d8005a80 con 0x7f890015d580 2023-11-24T06:28:32.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.592+0000 7f88e7fff640 1 -- 172.21.15.64:0/1052202383 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88f4037020 con 0x7f890015d580 2023-11-24T06:28:32.595 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:32.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.594+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f88ec06ae90 msgr2=0x7f88ec06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.594+0000 7f88e67fc640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f88ec06ae90 0x7f88ec06d350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88f0007fd0 tx=0x7f88f0009040 comp rx=0 tx=0).stop 2023-11-24T06:28:32.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.594+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890015d580 msgr2=0x7f89000ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.594+0000 7f88e67fc640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890015d580 0x7f89000ca110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f88f40098f0 tx=0x7f88f40046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 shutdown_connections 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f88ec06ae90 0x7f88ec06d350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f890015d960 0x7f89000ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890015d580 0x7f89000ca110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 --2- 172.21.15.64:0/1052202383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89000b20d0 0x7f89000b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 >> 172.21.15.64:0/1052202383 conn(0x7f89000d3040 msgr2=0x7f89001e4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 shutdown_connections 2023-11-24T06:28:32.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.595+0000 7f88e67fc640 1 -- 172.21.15.64:0/1052202383 wait complete. 2023-11-24T06:28:32.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:32.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.836+0000 7f7462ef8640 1 Processor -- start 2023-11-24T06:28:32.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.836+0000 7f7462ef8640 1 -- start start 2023-11-24T06:28:32.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.837+0000 7f7462ef8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160a20 0x7f745c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.837+0000 7f7462ef8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f745c0b3b20 0x7f745c0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.837+0000 7f7462ef8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745c0d1e00 con 0x7f745c160e00 2023-11-24T06:28:32.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.837+0000 7f7462ef8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745c0d3c40 con 0x7f745c0b3b20 2023-11-24T06:28:32.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.837+0000 7f7462ef8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0d3e50 con 0x7f745c160a20 2023-11-24T06:28:32.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.838+0000 7f7461ef6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160a20 0x7f745c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.838+0000 7f7461ef6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160a20 0x7f745c0cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:58180/0 (socket says 172.21.15.64:58180) 2023-11-24T06:28:32.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.838+0000 7f74626f7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f745c0b3b20 0x7f745c0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.838+0000 7f7461ef6640 1 -- 172.21.15.64:0/1299527010 learned_addr learned my addr 172.21.15.64:0/1299527010 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:32.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.838+0000 7f74626f7640 1 -- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160a20 msgr2=0x7f745c0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.838+0000 7f74626f7640 1 --2- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160a20 0x7f745c0cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.838+0000 7f74626f7640 1 -- 172.21.15.64:0/1299527010 >> v1:172.21.15.162:6789/0 conn(0x7f745c160e00 legacy=0x7f745c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.839+0000 7f74626f7640 1 -- 172.21.15.64:0/1299527010 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0d1a90 con 0x7f745c0b3b20 2023-11-24T06:28:32.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.839+0000 7f7461ef6640 1 --2- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160a20 0x7f745c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:32.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.840+0000 7f74626f7640 1 --2- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 0x7f745c0cfd90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7454009d90 tx=0x7f745402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d49db3b628a8a0c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.840+0000 7f7460ef4640 1 -- 172.21.15.64:0/1299527010 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745402cc50 con 0x7f745c0b3b20 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.840+0000 7f7460ef4640 1 -- 172.21.15.64:0/1299527010 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745402cdb0 con 0x7f745c0b3b20 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.840+0000 7f7460ef4640 1 -- 172.21.15.64:0/1299527010 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454034660 con 0x7f745c0b3b20 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.841+0000 7f7462ef8640 1 -- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 msgr2=0x7f745c0cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.841+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 0x7f745c0cfd90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7454009d90 tx=0x7f745402bea0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.841+0000 7f7462ef8640 1 -- 172.21.15.64:0/1299527010 shutdown_connections 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.841+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 0x7f745c0cfd90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.841+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1299527010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160a20 0x7f745c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.841+0000 7f7462ef8640 1 -- 172.21.15.64:0/1299527010 >> 172.21.15.64:0/1299527010 conn(0x7f745c0d3120 msgr2=0x7f745c0d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.841+0000 7f7462ef8640 1 -- 172.21.15.64:0/1299527010 shutdown_connections 2023-11-24T06:28:32.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.842+0000 7f7462ef8640 1 -- 172.21.15.64:0/1299527010 wait complete. 2023-11-24T06:28:32.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.843+0000 7f7462ef8640 1 Processor -- start 2023-11-24T06:28:32.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.843+0000 7f7462ef8640 1 -- start start 2023-11-24T06:28:32.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.845+0000 7f7462ef8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 0x7f745c14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.845+0000 7f7462ef8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f745c160a20 0x7f745c0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.845+0000 7f7462ef8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160e00 0x7f745c1f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.845+0000 7f7462ef8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0c6de0 con 0x7f745c160a20 2023-11-24T06:28:32.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.845+0000 7f7462ef8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c1f2ed0 con 0x7f745c0b3b20 2023-11-24T06:28:32.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.845+0000 7f7462ef8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c14d650 con 0x7f745c160e00 2023-11-24T06:28:32.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.845+0000 7f74616f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f745c160a20 0x7f745c0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f7461ef6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 0x7f745c14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f745c160a20 0x7f745c0ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43744/0 (socket says 172.21.15.64:43744) 2023-11-24T06:28:32.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 -- 172.21.15.64:0/1676196084 learned_addr learned my addr 172.21.15.64:0/1676196084 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:32.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74626f7640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160e00 0x7f745c1f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 -- 172.21.15.64:0/1676196084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160e00 msgr2=0x7f745c1f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160e00 0x7f745c1f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 -- 172.21.15.64:0/1676196084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 msgr2=0x7f745c14d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 0x7f745c14d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 -- 172.21.15.64:0/1676196084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580094d0 con 0x7f745c160a20 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.846+0000 7f74616f5640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f745c160a20 0x7f745c0ccaf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f744c00c3c0 tx=0x7f744c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.847+0000 7f74527fc640 1 -- 172.21.15.64:0/1676196084 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c0052d0 con 0x7f745c160a20 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.847+0000 7f74527fc640 1 -- 172.21.15.64:0/1676196084 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c005450 con 0x7f745c160a20 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.847+0000 7f74527fc640 1 -- 172.21.15.64:0/1676196084 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c003830 con 0x7f745c160a20 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.847+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7454009a70 con 0x7f745c160a20 2023-11-24T06:28:32.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.847+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c0b87c0 con 0x7f745c160a20 2023-11-24T06:28:32.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.848+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c067b60 con 0x7f745c160a20 2023-11-24T06:28:32.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.850+0000 7f74527fc640 1 -- 172.21.15.64:0/1676196084 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f744c003990 con 0x7f745c160a20 2023-11-24T06:28:32.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.851+0000 7f74527fc640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f744006af10 0x7f744006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.852+0000 7f74527fc640 1 -- 172.21.15.64:0/1676196084 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f744c088de0 con 0x7f745c160a20 2023-11-24T06:28:32.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.853+0000 7f7461ef6640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f744006af10 0x7f744006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.854+0000 7f7461ef6640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f744006af10 0x7f744006d3d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7458006ca0 tx=0x7f7458009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:32.855+0000 7f74527fc640 1 -- 172.21.15.64:0/1676196084 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f744c053ec0 con 0x7f745c160a20 2023-11-24T06:28:33.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.131+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f745c160220 con 0x7f745c160a20 2023-11-24T06:28:33.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.132+0000 7f74527fc640 1 -- 172.21.15.64:0/1676196084 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f744c0574e0 con 0x7f745c160a20 2023-11-24T06:28:33.135 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f744006af10 msgr2=0x7f744006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f744006af10 0x7f744006d3d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7458006ca0 tx=0x7f7458009240 comp rx=0 tx=0).stop 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f745c160a20 msgr2=0x7f745c0ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f745c160a20 0x7f745c0ccaf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f744c00c3c0 tx=0x7f744c00c880 comp rx=0 tx=0).stop 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 shutdown_connections 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f744006af10 0x7f744006d3d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f745c160e00 0x7f745c1f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f745c160a20 0x7f745c0ccaf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 --2- 172.21.15.64:0/1676196084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b3b20 0x7f745c14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 >> 172.21.15.64:0/1676196084 conn(0x7f745c0d3120 msgr2=0x7f745c0cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:33.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 shutdown_connections 2023-11-24T06:28:33.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.135+0000 7f7462ef8640 1 -- 172.21.15.64:0/1676196084 wait complete. 2023-11-24T06:28:33.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:33.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.374+0000 7f094285c640 1 Processor -- start 2023-11-24T06:28:33.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.374+0000 7f094285c640 1 -- start start 2023-11-24T06:28:33.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.376+0000 7f094285c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c160f80 0x7f093c161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.376+0000 7f094285c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f093c0ced50 0x7f093c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.376+0000 7f094285c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0cf1b0 con 0x7f093c0ce8f0 2023-11-24T06:28:33.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.376+0000 7f094285c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c1618a0 con 0x7f093c0ced50 2023-11-24T06:28:33.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.376+0000 7f094285c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0d0de0 con 0x7f093c160f80 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f093b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f093c0ce8f0 0x7f093c0cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41612/0 (socket says 172.21.15.64:41612) 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f093b7fe640 1 -- 172.21.15.64:0/1819663500 learned_addr learned my addr 172.21.15.64:0/1819663500 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f0940df9640 1 --2- 172.21.15.64:0/1819663500 >> v2:172.21.15.162:3300/0 conn(0x7f093c0ced50 0x7f093c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f093bfff640 1 --2- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c160f80 0x7f093c161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f0940df9640 1 -- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c160f80 msgr2=0x7f093c161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f0940df9640 1 --2- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c160f80 0x7f093c161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f0940df9640 1 -- 172.21.15.64:0/1819663500 >> v1:172.21.15.64:6789/0 conn(0x7f093c0ce8f0 legacy=0x7f093c0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f093affd640 1 -- 172.21.15.64:0/1819663500 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637515807 0 0) 0x7f093c0cf1b0 con 0x7f093c0ce8f0 2023-11-24T06:28:33.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.377+0000 7f0940df9640 1 -- 172.21.15.64:0/1819663500 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0d10c0 con 0x7f093c0ced50 2023-11-24T06:28:33.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.379+0000 7f0940df9640 1 --2- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c0ced50 0x7f093c0d02f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0934009d90 tx=0x7f093402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565d132117b2c9b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:33.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.379+0000 7f093affd640 1 -- 172.21.15.64:0/1819663500 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093402cc50 con 0x7f093c0ced50 2023-11-24T06:28:33.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.379+0000 7f093affd640 1 -- 172.21.15.64:0/1819663500 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093402cdb0 con 0x7f093c0ced50 2023-11-24T06:28:33.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.379+0000 7f093affd640 1 -- 172.21.15.64:0/1819663500 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934034660 con 0x7f093c0ced50 2023-11-24T06:28:33.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 -- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c0ced50 msgr2=0x7f093c0d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 --2- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c0ced50 0x7f093c0d02f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0934009d90 tx=0x7f093402bea0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 -- 172.21.15.64:0/1819663500 shutdown_connections 2023-11-24T06:28:33.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 --2- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c0ced50 0x7f093c0d02f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 --2- 172.21.15.64:0/1819663500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c160f80 0x7f093c161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 -- 172.21.15.64:0/1819663500 >> 172.21.15.64:0/1819663500 conn(0x7f093c09c550 msgr2=0x7f093c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:33.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 -- 172.21.15.64:0/1819663500 shutdown_connections 2023-11-24T06:28:33.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.380+0000 7f094285c640 1 -- 172.21.15.64:0/1819663500 wait complete. 2023-11-24T06:28:33.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.383+0000 7f094285c640 1 Processor -- start 2023-11-24T06:28:33.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.383+0000 7f094285c640 1 -- start start 2023-11-24T06:28:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f094285c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0ce8f0 0x7f093c160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f094285c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c0ced50 0x7f093c14e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f094285c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c160f80 0x7f093c15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f094285c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c1f47c0 con 0x7f093c0ce8f0 2023-11-24T06:28:33.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f094285c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c1f4300 con 0x7f093c160f80 2023-11-24T06:28:33.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f094285c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c06c200 con 0x7f093c0ced50 2023-11-24T06:28:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f0940df9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c160f80 0x7f093c15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f0940df9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c160f80 0x7f093c15dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:54712/0 (socket says 172.21.15.64:54712) 2023-11-24T06:28:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f0940df9640 1 -- 172.21.15.64:0/1487843679 learned_addr learned my addr 172.21.15.64:0/1487843679 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.385+0000 7f093bfff640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0ce8f0 0x7f093c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f093b7fe640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c0ced50 0x7f093c14e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f0940df9640 1 -- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c0ced50 msgr2=0x7f093c14e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f0940df9640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c0ced50 0x7f093c14e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f0940df9640 1 -- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0ce8f0 msgr2=0x7f093c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f093b7fe640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c0ced50 0x7f093c14e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f0940df9640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0ce8f0 0x7f093c160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f0940df9640 1 -- 172.21.15.64:0/1487843679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0934009a70 con 0x7f093c160f80 2023-11-24T06:28:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.386+0000 7f093bfff640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0ce8f0 0x7f093c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.387+0000 7f0940df9640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c160f80 0x7f093c15dd40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0934009d90 tx=0x7f0934004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.387+0000 7f0938ff9640 1 -- 172.21.15.64:0/1487843679 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934040650 con 0x7f093c160f80 2023-11-24T06:28:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.387+0000 7f0938ff9640 1 -- 172.21.15.64:0/1487843679 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09340407b0 con 0x7f093c160f80 2023-11-24T06:28:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.387+0000 7f0938ff9640 1 -- 172.21.15.64:0/1487843679 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934040b60 con 0x7f093c160f80 2023-11-24T06:28:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.388+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c06c4e0 con 0x7f093c160f80 2023-11-24T06:28:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.388+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c1f3e00 con 0x7f093c160f80 2023-11-24T06:28:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.389+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c1f4900 con 0x7f093c160f80 2023-11-24T06:28:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.391+0000 7f0938ff9640 1 -- 172.21.15.64:0/1487843679 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0934040cc0 con 0x7f093c160f80 2023-11-24T06:28:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.392+0000 7f0938ff9640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f092806ae40 0x7f092806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.392+0000 7f0938ff9640 1 -- 172.21.15.64:0/1487843679 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f09340ac500 con 0x7f093c160f80 2023-11-24T06:28:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.392+0000 7f093bfff640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f092806ae40 0x7f092806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.393+0000 7f093bfff640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f092806ae40 0x7f092806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f093c15d9e0 tx=0x7f092c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.410+0000 7f0938ff9640 1 -- 172.21.15.64:0/1487843679 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0934077700 con 0x7f093c160f80 2023-11-24T06:28:33.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.759+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f093c1f4900 con 0x7f093c160f80 2023-11-24T06:28:33.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.760+0000 7f0938ff9640 1 -- 172.21.15.64:0/1487843679 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f093c1f4900 con 0x7f093c160f80 2023-11-24T06:28:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.762+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f092806ae40 msgr2=0x7f092806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.762+0000 7f094285c640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f092806ae40 0x7f092806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f093c15d9e0 tx=0x7f092c009190 comp rx=0 tx=0).stop 2023-11-24T06:28:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c160f80 msgr2=0x7f093c15dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c160f80 0x7f093c15dd40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0934009d90 tx=0x7f0934004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 shutdown_connections 2023-11-24T06:28:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f092806ae40 0x7f092806d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f093c160f80 0x7f093c15dd40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f093c0ced50 0x7f093c14e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 --2- 172.21.15.64:0/1487843679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0ce8f0 0x7f093c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 >> 172.21.15.64:0/1487843679 conn(0x7f093c09c550 msgr2=0x7f093c06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:33.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 shutdown_connections 2023-11-24T06:28:33.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.763+0000 7f094285c640 1 -- 172.21.15.64:0/1487843679 wait complete. 2023-11-24T06:28:33.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.991+0000 7fdd0e42e640 1 Processor -- start 2023-11-24T06:28:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.991+0000 7fdd0e42e640 1 -- start start 2023-11-24T06:28:33.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.993+0000 7fdd0e42e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd080b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.993+0000 7fdd0e42e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd080cf9f0 0x7fdd080cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.993+0000 7fdd0e42e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd080d0a10 con 0x7fdd08161010 2023-11-24T06:28:33.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.993+0000 7fdd0e42e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd080d03b0 con 0x7fdd080cf9f0 2023-11-24T06:28:33.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.993+0000 7fdd0e42e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080d1f50 con 0x7fdd08160920 2023-11-24T06:28:33.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0cc2b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd080cf9f0 0x7fdd080cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0dc2d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdd08161010 0x7fdd081614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41614/0 (socket says 172.21.15.64:41614) 2023-11-24T06:28:33.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0dc2d640 1 -- 172.21.15.64:0/3478498666 learned_addr learned my addr 172.21.15.64:0/3478498666 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:33.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0d42c640 1 --2- 172.21.15.64:0/3478498666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd080b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0cc2b640 1 -- 172.21.15.64:0/3478498666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 msgr2=0x7fdd080b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:33.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0cc2b640 1 --2- 172.21.15.64:0/3478498666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd080b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0cc2b640 1 -- 172.21.15.64:0/3478498666 >> v1:172.21.15.64:6789/0 conn(0x7fdd08161010 legacy=0x7fdd081614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdd0cc2b640 1 -- 172.21.15.64:0/3478498666 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080cf080 con 0x7fdd080cf9f0 2023-11-24T06:28:33.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.994+0000 7fdcfffff640 1 -- 172.21.15.64:0/3478498666 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808612300 0 0) 0x7fdd080d0a10 con 0x7fdd08161010 2023-11-24T06:28:33.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.996+0000 7fdd0cc2b640 1 --2- 172.21.15.64:0/3478498666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd080cf9f0 0x7fdd080cfe70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80097a0 tx=0x7fdcf802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3969232967045f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:33.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.996+0000 7fdcfffff640 1 -- 172.21.15.64:0/3478498666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8031830 con 0x7fdd080cf9f0 2023-11-24T06:28:33.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.996+0000 7fdcfffff640 1 -- 172.21.15.64:0/3478498666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf8031990 con 0x7fdd080cf9f0 2023-11-24T06:28:33.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.996+0000 7fdcfffff640 1 -- 172.21.15.64:0/3478498666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8031d20 con 0x7fdd080cf9f0 2023-11-24T06:28:33.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 -- 172.21.15.64:0/3478498666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd080cf9f0 msgr2=0x7fdd080cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/3478498666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd080cf9f0 0x7fdd080cfe70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80097a0 tx=0x7fdcf802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 -- 172.21.15.64:0/3478498666 shutdown_connections 2023-11-24T06:28:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/3478498666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd080cf9f0 0x7fdd080cfe70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/3478498666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd080b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 -- 172.21.15.64:0/3478498666 >> 172.21.15.64:0/3478498666 conn(0x7fdd080d3090 msgr2=0x7fdd080d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 -- 172.21.15.64:0/3478498666 shutdown_connections 2023-11-24T06:28:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.997+0000 7fdd0e42e640 1 -- 172.21.15.64:0/3478498666 wait complete. 2023-11-24T06:28:34.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.999+0000 7fdd0e42e640 1 Processor -- start 2023-11-24T06:28:34.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:33.999+0000 7fdd0e42e640 1 -- start start 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0e42e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd080cf9f0 0x7fdd0815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0e42e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd081f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0e42e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd08161010 0x7fdd080cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0e42e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd081f3c60 con 0x7fdd08161010 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0e42e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080ca160 con 0x7fdd080cf9f0 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0e42e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080ca400 con 0x7fdd08160920 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0dc2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd08161010 0x7fdd080cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0dc2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd08161010 0x7fdd080cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40208/0 (socket says 172.21.15.64:40208) 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0dc2d640 1 -- 172.21.15.64:0/2123187068 learned_addr learned my addr 172.21.15.64:0/2123187068 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0cc2b640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd081f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.001+0000 7fdd0d42c640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd080cf9f0 0x7fdd0815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdd0dc2d640 1 -- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 msgr2=0x7fdd081f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdd0dc2d640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd081f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdd0dc2d640 1 -- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd080cf9f0 msgr2=0x7fdd0815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdd0dc2d640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd080cf9f0 0x7fdd0815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdd0dc2d640 1 -- 172.21.15.64:0/2123187068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8009400 con 0x7fdd08161010 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdd0dc2d640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd08161010 0x7fdd080cb4a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000c700 tx=0x7fdd0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdcfdffb640 1 -- 172.21.15.64:0/2123187068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000ce10 con 0x7fdd08161010 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdcfdffb640 1 -- 172.21.15.64:0/2123187068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0000a640 con 0x7fdd08161010 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.002+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd080ca6e0 con 0x7fdd08161010 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.003+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd080caca0 con 0x7fdd08161010 2023-11-24T06:28:34.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.003+0000 7fdcfdffb640 1 -- 172.21.15.64:0/2123187068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000aa70 con 0x7fdd08161010 2023-11-24T06:28:34.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.004+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce8005250 con 0x7fdd08161010 2023-11-24T06:28:34.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.005+0000 7fdcfdffb640 1 -- 172.21.15.64:0/2123187068 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd0000abd0 con 0x7fdd08161010 2023-11-24T06:28:34.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.007+0000 7fdcfdffb640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcec06ae40 0x7fdcec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.007+0000 7fdcfdffb640 1 -- 172.21.15.64:0/2123187068 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fdd00088db0 con 0x7fdd08161010 2023-11-24T06:28:34.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.008+0000 7fdd0d42c640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcec06ae40 0x7fdcec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.009+0000 7fdd0d42c640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcec06ae40 0x7fdcec06d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd04006fd0 tx=0x7fdd04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:34.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.014+0000 7fdcfdffb640 1 -- 172.21.15.64:0/2123187068 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd0008d070 con 0x7fdd08161010 2023-11-24T06:28:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.297+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdce8005a80 con 0x7fdd08161010 2023-11-24T06:28:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.298+0000 7fdcfdffb640 1 -- 172.21.15.64:0/2123187068 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd00018020 con 0x7fdd08161010 2023-11-24T06:28:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcec06ae40 msgr2=0x7fdcec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcec06ae40 0x7fdcec06d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd04006fd0 tx=0x7fdd04008040 comp rx=0 tx=0).stop 2023-11-24T06:28:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd08161010 msgr2=0x7fdd080cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd08161010 0x7fdd080cb4a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000c700 tx=0x7fdd0000cbc0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 shutdown_connections 2023-11-24T06:28:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcec06ae40 0x7fdcec06d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd08161010 0x7fdd080cb4a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd08160920 0x7fdd081f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 --2- 172.21.15.64:0/2123187068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd080cf9f0 0x7fdd0815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 >> 172.21.15.64:0/2123187068 conn(0x7fdd080d3090 msgr2=0x7fdd08161420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 shutdown_connections 2023-11-24T06:28:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.301+0000 7fdd0e42e640 1 -- 172.21.15.64:0/2123187068 wait complete. 2023-11-24T06:28:34.324 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T06:28:34.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:34.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:34.424 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:34.424 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:28:34.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:34.425 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:28:34.478 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:28:34.478 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:34.534 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:28:34.534 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:34.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:34.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.778+0000 7f1eea864640 1 Processor -- start 2023-11-24T06:28:34.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.779+0000 7f1eea864640 1 -- start start 2023-11-24T06:28:34.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.780+0000 7f1eea864640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ee413ff20 0x7f1ee415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.780+0000 7f1eea864640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ee4140300 0x7f1ee406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.780+0000 7f1eea864640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee406a920 0x7f1ee40b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.780+0000 7f1eea864640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee406adb0 con 0x7f1ee4140300 2023-11-24T06:28:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.780+0000 7f1eea864640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee40b5440 con 0x7f1ee413ff20 2023-11-24T06:28:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.780+0000 7f1eea864640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee4161c80 con 0x7f1ee406a920 2023-11-24T06:28:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ee4140300 0x7f1ee406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ee4140300 0x7f1ee406a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40218/0 (socket says 172.21.15.64:40218) 2023-11-24T06:28:34.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 -- 172.21.15.64:0/3144570352 learned_addr learned my addr 172.21.15.64:0/3144570352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:34.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9862640 1 --2- 172.21.15.64:0/3144570352 >> v2:172.21.15.162:3300/0 conn(0x7f1ee413ff20 0x7f1ee415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1eea063640 1 --2- 172.21.15.64:0/3144570352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee406a920 0x7f1ee40b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 -- 172.21.15.64:0/3144570352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee406a920 msgr2=0x7f1ee40b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:34.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 --2- 172.21.15.64:0/3144570352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee406a920 0x7f1ee40b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 -- 172.21.15.64:0/3144570352 >> v2:172.21.15.162:3300/0 conn(0x7f1ee413ff20 msgr2=0x7f1ee415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:34.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 --2- 172.21.15.64:0/3144570352 >> v2:172.21.15.162:3300/0 conn(0x7f1ee413ff20 0x7f1ee415e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9061640 1 -- 172.21.15.64:0/3144570352 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee415e6f0 con 0x7f1ee4140300 2023-11-24T06:28:34.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.781+0000 7f1ee9862640 1 --2- 172.21.15.64:0/3144570352 >> v2:172.21.15.162:3300/0 conn(0x7f1ee413ff20 0x7f1ee415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:34.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.783+0000 7f1ee9061640 1 --2- 172.21.15.64:0/3144570352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee406a3e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009870 tx=0x7f1ed402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc172b1eb9d0226 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:34.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.783+0000 7f1ee8860640 1 -- 172.21.15.64:0/3144570352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40317c0 con 0x7f1ee4140300 2023-11-24T06:28:34.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.783+0000 7f1ee8860640 1 -- 172.21.15.64:0/3144570352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed4031920 con 0x7f1ee4140300 2023-11-24T06:28:34.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.783+0000 7f1ee8860640 1 -- 172.21.15.64:0/3144570352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4031cb0 con 0x7f1ee4140300 2023-11-24T06:28:34.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 -- 172.21.15.64:0/3144570352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 msgr2=0x7f1ee406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 --2- 172.21.15.64:0/3144570352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee406a3e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009870 tx=0x7f1ed402b860 comp rx=0 tx=0).stop 2023-11-24T06:28:34.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 -- 172.21.15.64:0/3144570352 shutdown_connections 2023-11-24T06:28:34.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 --2- 172.21.15.64:0/3144570352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee406a920 0x7f1ee40b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 --2- 172.21.15.64:0/3144570352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee406a3e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 --2- 172.21.15.64:0/3144570352 >> v2:172.21.15.162:3300/0 conn(0x7f1ee413ff20 0x7f1ee415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 -- 172.21.15.64:0/3144570352 >> 172.21.15.64:0/3144570352 conn(0x7f1ee40b1cd0 msgr2=0x7f1ee40b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:34.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 -- 172.21.15.64:0/3144570352 shutdown_connections 2023-11-24T06:28:34.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.784+0000 7f1eea864640 1 -- 172.21.15.64:0/3144570352 wait complete. 2023-11-24T06:28:34.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.787+0000 7f1eea864640 1 Processor -- start 2023-11-24T06:28:34.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.787+0000 7f1eea864640 1 -- start start 2023-11-24T06:28:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea864640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee406a920 0x7f1ee41597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea864640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee413ff20 0x7f1ee4156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea864640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea864640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee4159d00 con 0x7f1ee4140300 2023-11-24T06:28:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea864640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee41f2e30 con 0x7f1ee406a920 2023-11-24T06:28:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea864640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee41f3110 con 0x7f1ee413ff20 2023-11-24T06:28:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea063640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea063640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee415cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40222/0 (socket says 172.21.15.64:40222) 2023-11-24T06:28:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.788+0000 7f1eea063640 1 -- 172.21.15.64:0/3171185687 learned_addr learned my addr 172.21.15.64:0/3171185687 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:34.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eea063640 1 -- 172.21.15.64:0/3171185687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee413ff20 msgr2=0x7f1ee4156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eea063640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee413ff20 0x7f1ee4156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eea063640 1 -- 172.21.15.64:0/3171185687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee406a920 msgr2=0x7f1ee41597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:34.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1ee9862640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee406a920 0x7f1ee41597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eea063640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee406a920 0x7f1ee41597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:34.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eea063640 1 -- 172.21.15.64:0/3171185687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40094d0 con 0x7f1ee4140300 2023-11-24T06:28:34.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eea063640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee415cab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007e40 tx=0x7f1edc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:34.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eda7fc640 1 -- 172.21.15.64:0/3171185687 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc014620 con 0x7f1ee4140300 2023-11-24T06:28:34.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eda7fc640 1 -- 172.21.15.64:0/3171185687 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00ee90 con 0x7f1ee4140300 2023-11-24T06:28:34.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eda7fc640 1 -- 172.21.15.64:0/3171185687 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0149e0 con 0x7f1ee4140300 2023-11-24T06:28:34.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.789+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee41f33f0 con 0x7f1ee4140300 2023-11-24T06:28:34.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.790+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee41f39b0 con 0x7f1ee4140300 2023-11-24T06:28:34.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.791+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee415b1c0 con 0x7f1ee4140300 2023-11-24T06:28:34.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.792+0000 7f1eda7fc640 1 -- 172.21.15.64:0/3171185687 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1edc014b40 con 0x7f1ee4140300 2023-11-24T06:28:34.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.793+0000 7f1eda7fc640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1ec806ae40 0x7f1ec806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:34.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.793+0000 7f1eda7fc640 1 -- 172.21.15.64:0/3171185687 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f1edc08a000 con 0x7f1ee4140300 2023-11-24T06:28:34.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.794+0000 7f1ee9862640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1ec806ae40 0x7f1ec806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:34.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.795+0000 7f1ee9862640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1ec806ae40 0x7f1ec806d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006fd0 tx=0x7f1ee0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:34.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:34.798+0000 7f1eda7fc640 1 -- 172.21.15.64:0/3171185687 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ee415b1c0 con 0x7f1ee4140300 2023-11-24T06:28:35.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.044+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee415b1c0 con 0x7f1ee4140300 2023-11-24T06:28:35.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.046+0000 7f1eda7fc640 1 -- 172.21.15.64:0/3171185687 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f1ee415b1c0 con 0x7f1ee4140300 2023-11-24T06:28:35.050 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:35.050 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:35.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.049+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1ec806ae40 msgr2=0x7f1ec806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.049+0000 7f1eea864640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1ec806ae40 0x7f1ec806d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006fd0 tx=0x7f1ee0008040 comp rx=0 tx=0).stop 2023-11-24T06:28:35.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 msgr2=0x7f1ee415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee415cab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007e40 tx=0x7f1edc00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 shutdown_connections 2023-11-24T06:28:35.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1ec806ae40 0x7f1ec806d300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ee4140300 0x7f1ee415cab0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee413ff20 0x7f1ee4156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 --2- 172.21.15.64:0/3171185687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee406a920 0x7f1ee41597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 >> 172.21.15.64:0/3171185687 conn(0x7f1ee40b1cd0 msgr2=0x7f1ee413e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:35.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 shutdown_connections 2023-11-24T06:28:35.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.050+0000 7f1eea864640 1 -- 172.21.15.64:0/3171185687 wait complete. 2023-11-24T06:28:35.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:35.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:35.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.292+0000 7f535e7d5640 1 Processor -- start 2023-11-24T06:28:35.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.292+0000 7f535e7d5640 1 -- start start 2023-11-24T06:28:35.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.293+0000 7f535e7d5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5358160f80 0x7f5358161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f535e7d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f53580ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f535e7d5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53581618a0 con 0x7f53580cfbe0 2023-11-24T06:28:35.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f535e7d5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53580d0de0 con 0x7f5358160f80 2023-11-24T06:28:35.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f535e7d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53580d10c0 con 0x7f53580ce8f0 2023-11-24T06:28:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f53577fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f53580ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f5357fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5358160f80 0x7f5358161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f53577fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f53580ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44118/0 (socket says 172.21.15.64:44118) 2023-11-24T06:28:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f53577fe640 1 -- 172.21.15.64:0/1747242069 learned_addr learned my addr 172.21.15.64:0/1747242069 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f5357fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5358160f80 0x7f5358161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:54736/0 (socket says 172.21.15.64:54736) 2023-11-24T06:28:35.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.294+0000 7f535cd72640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f53580cfbe0 0x7f53580d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:49428/0 (socket says 172.21.15.64:49428) 2023-11-24T06:28:35.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.295+0000 7f53577fe640 1 -- 172.21.15.64:0/1747242069 >> v2:172.21.15.162:3300/0 conn(0x7f5358160f80 msgr2=0x7f5358161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:35.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.295+0000 7f53577fe640 1 --2- 172.21.15.64:0/1747242069 >> v2:172.21.15.162:3300/0 conn(0x7f5358160f80 0x7f5358161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.295+0000 7f53577fe640 1 -- 172.21.15.64:0/1747242069 >> v1:172.21.15.162:6789/0 conn(0x7f53580cfbe0 legacy=0x7f53580d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.295+0000 7f53577fe640 1 -- 172.21.15.64:0/1747242069 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53580d13a0 con 0x7f53580ce8f0 2023-11-24T06:28:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.296+0000 7f53577fe640 1 --2- 172.21.15.64:0/1747242069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f53580ced50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5a2d574149efb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f5356ffd640 1 -- 172.21.15.64:0/1747242069 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c030840 con 0x7f53580ce8f0 2023-11-24T06:28:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f5356ffd640 1 -- 172.21.15.64:0/1747242069 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c0309a0 con 0x7f53580ce8f0 2023-11-24T06:28:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f5356ffd640 1 -- 172.21.15.64:0/1747242069 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c030d30 con 0x7f53580ce8f0 2023-11-24T06:28:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f535e7d5640 1 -- 172.21.15.64:0/1747242069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 msgr2=0x7f53580ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f535e7d5640 1 --2- 172.21.15.64:0/1747242069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f53580ced50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02b980 comp rx=0 tx=0).stop 2023-11-24T06:28:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f535e7d5640 1 -- 172.21.15.64:0/1747242069 shutdown_connections 2023-11-24T06:28:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f535e7d5640 1 --2- 172.21.15.64:0/1747242069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f53580ced50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f535e7d5640 1 --2- 172.21.15.64:0/1747242069 >> v2:172.21.15.162:3300/0 conn(0x7f5358160f80 0x7f5358161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f535e7d5640 1 -- 172.21.15.64:0/1747242069 >> 172.21.15.64:0/1747242069 conn(0x7f535809c550 msgr2=0x7f53580d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.297+0000 7f535e7d5640 1 -- 172.21.15.64:0/1747242069 shutdown_connections 2023-11-24T06:28:35.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.298+0000 7f535e7d5640 1 -- 172.21.15.64:0/1747242069 wait complete. 2023-11-24T06:28:35.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.299+0000 7f535e7d5640 1 Processor -- start 2023-11-24T06:28:35.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.300+0000 7f535e7d5640 1 -- start start 2023-11-24T06:28:35.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.301+0000 7f535e7d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f535815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.301+0000 7f535e7d5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53580cfbe0 0x7f535815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.301+0000 7f535e7d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5358160f80 0x7f535815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.301+0000 7f535e7d5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535815e030 con 0x7f53580cfbe0 2023-11-24T06:28:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.301+0000 7f535e7d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535815e310 con 0x7f5358160f80 2023-11-24T06:28:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.301+0000 7f535e7d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535815e5f0 con 0x7f53580ce8f0 2023-11-24T06:28:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.301+0000 7f53577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53580cfbe0 0x7f535815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f53577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53580cfbe0 0x7f535815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40224/0 (socket says 172.21.15.64:40224) 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f53577fe640 1 -- 172.21.15.64:0/2025953812 learned_addr learned my addr 172.21.15.64:0/2025953812 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f5357fff640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f535815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f535cd72640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5358160f80 0x7f535815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f53577fe640 1 -- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 msgr2=0x7f535815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f53577fe640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f535815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f53577fe640 1 -- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5358160f80 msgr2=0x7f535815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f53577fe640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5358160f80 0x7f535815f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.302+0000 7f53577fe640 1 -- 172.21.15.64:0/2025953812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c0094d0 con 0x7f53580cfbe0 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.303+0000 7f53577fe640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53580cfbe0 0x7f535815d6c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f534c000c00 tx=0x7f534c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.303+0000 7f5354ff9640 1 -- 172.21.15.64:0/2025953812 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c039a90 con 0x7f53580cfbe0 2023-11-24T06:28:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.303+0000 7f5354ff9640 1 -- 172.21.15.64:0/2025953812 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c039bf0 con 0x7f53580cfbe0 2023-11-24T06:28:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.303+0000 7f5354ff9640 1 -- 172.21.15.64:0/2025953812 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c043410 con 0x7f53580cfbe0 2023-11-24T06:28:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.303+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535815e8d0 con 0x7f53580cfbe0 2023-11-24T06:28:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.303+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535815dc00 con 0x7f53580cfbe0 2023-11-24T06:28:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.305+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53580675d0 con 0x7f53580cfbe0 2023-11-24T06:28:35.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.309+0000 7f5354ff9640 1 -- 172.21.15.64:0/2025953812 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f534c02f390 con 0x7f53580cfbe0 2023-11-24T06:28:35.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.310+0000 7f5354ff9640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f534406ae40 0x7f534406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.310+0000 7f5354ff9640 1 -- 172.21.15.64:0/2025953812 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f534c0abb10 con 0x7f53580cfbe0 2023-11-24T06:28:35.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.311+0000 7f5357fff640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f534406ae40 0x7f534406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.312+0000 7f5357fff640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f534406ae40 0x7f534406d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5348009b40 tx=0x7f5348009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.312+0000 7f5354ff9640 1 -- 172.21.15.64:0/2025953812 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f534c076c80 con 0x7f53580cfbe0 2023-11-24T06:28:35.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.574+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53580d3920 con 0x7f53580cfbe0 2023-11-24T06:28:35.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.575+0000 7f5354ff9640 1 -- 172.21.15.64:0/2025953812 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f534c034020 con 0x7f53580cfbe0 2023-11-24T06:28:35.578 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f534406ae40 msgr2=0x7f534406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f534406ae40 0x7f534406d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5348009b40 tx=0x7f5348009190 comp rx=0 tx=0).stop 2023-11-24T06:28:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53580cfbe0 msgr2=0x7f535815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53580cfbe0 0x7f535815d6c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f534c000c00 tx=0x7f534c004730 comp rx=0 tx=0).stop 2023-11-24T06:28:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 shutdown_connections 2023-11-24T06:28:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f534406ae40 0x7f534406d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5358160f80 0x7f535815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53580cfbe0 0x7f535815d6c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 --2- 172.21.15.64:0/2025953812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53580ce8f0 0x7f535815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 >> 172.21.15.64:0/2025953812 conn(0x7f535809c550 msgr2=0x7f53580d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 shutdown_connections 2023-11-24T06:28:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.578+0000 7f535e7d5640 1 -- 172.21.15.64:0/2025953812 wait complete. 2023-11-24T06:28:35.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:35.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.804+0000 7f1d567db640 1 Processor -- start 2023-11-24T06:28:35.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.804+0000 7f1d567db640 1 -- start start 2023-11-24T06:28:35.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.805+0000 7f1d567db640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d500b35a0 0x7f1d500cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.805+0000 7f1d567db640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d500ce440 con 0x7f1d5013fcb0 2023-11-24T06:28:35.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.805+0000 7f1d567db640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d500ce720 con 0x7f1d5013cf10 2023-11-24T06:28:35.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.805+0000 7f1d567db640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d500cea00 con 0x7f1d500b35a0 2023-11-24T06:28:35.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.805+0000 7f1d54fd8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d5013fcb0 0x7f1d500cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41620/0 (socket says 172.21.15.64:41620) 2023-11-24T06:28:35.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.805+0000 7f1d54fd8640 1 -- 172.21.15.64:0/4274159922 learned_addr learned my addr 172.21.15.64:0/4274159922 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:35.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.806+0000 7f1d55fda640 1 --2- 172.21.15.64:0/4274159922 >> v2:172.21.15.162:3300/0 conn(0x7f1d500b35a0 0x7f1d500cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.806+0000 7f1d47fff640 1 -- 172.21.15.64:0/4274159922 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525475074 0 0) 0x7f1d500ce720 con 0x7f1d5013cf10 2023-11-24T06:28:35.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.806+0000 7f1d47fff640 1 -- 172.21.15.64:0/4274159922 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d3c003520 con 0x7f1d5013cf10 2023-11-24T06:28:35.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.806+0000 7f1d55fda640 1 -- 172.21.15.64:0/4274159922 >> v1:172.21.15.162:6789/0 conn(0x7f1d5013cf10 legacy=0x7f1d5013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.806+0000 7f1d55fda640 1 -- 172.21.15.64:0/4274159922 >> v1:172.21.15.64:6789/0 conn(0x7f1d5013fcb0 legacy=0x7f1d500cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.806+0000 7f1d55fda640 1 -- 172.21.15.64:0/4274159922 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500b1d90 con 0x7f1d500b35a0 2023-11-24T06:28:35.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.808+0000 7f1d55fda640 1 --2- 172.21.15.64:0/4274159922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d500b35a0 0x7f1d500cddf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009d90 tx=0x7f1d4802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fe829aa26a9fa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.808+0000 7f1d47fff640 1 -- 172.21.15.64:0/4274159922 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4802ccf0 con 0x7f1d500b35a0 2023-11-24T06:28:35.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.808+0000 7f1d47fff640 1 -- 172.21.15.64:0/4274159922 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4802ce50 con 0x7f1d500b35a0 2023-11-24T06:28:35.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.808+0000 7f1d47fff640 1 -- 172.21.15.64:0/4274159922 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48034640 con 0x7f1d500b35a0 2023-11-24T06:28:35.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.809+0000 7f1d567db640 1 -- 172.21.15.64:0/4274159922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d500b35a0 msgr2=0x7f1d500cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.809+0000 7f1d567db640 1 --2- 172.21.15.64:0/4274159922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d500b35a0 0x7f1d500cddf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009d90 tx=0x7f1d4802bf40 comp rx=0 tx=0).stop 2023-11-24T06:28:35.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.809+0000 7f1d567db640 1 -- 172.21.15.64:0/4274159922 shutdown_connections 2023-11-24T06:28:35.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.809+0000 7f1d567db640 1 --2- 172.21.15.64:0/4274159922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d500b35a0 0x7f1d500cddf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.809+0000 7f1d567db640 1 -- 172.21.15.64:0/4274159922 >> 172.21.15.64:0/4274159922 conn(0x7f1d5006a7d0 msgr2=0x7f1d5006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:35.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.809+0000 7f1d567db640 1 -- 172.21.15.64:0/4274159922 shutdown_connections 2023-11-24T06:28:35.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.809+0000 7f1d567db640 1 -- 172.21.15.64:0/4274159922 wait complete. 2023-11-24T06:28:35.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.811+0000 7f1d567db640 1 Processor -- start 2023-11-24T06:28:35.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.811+0000 7f1d567db640 1 -- start start 2023-11-24T06:28:35.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.812+0000 7f1d567db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d500b35a0 0x7f1d501316c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.812+0000 7f1d567db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d5013cf10 0x7f1d5013ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.812+0000 7f1d567db640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d5013fcb0 0x7f1d5013d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.812+0000 7f1d567db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50131c00 con 0x7f1d5013cf10 2023-11-24T06:28:35.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.812+0000 7f1d567db640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50130c00 con 0x7f1d5013fcb0 2023-11-24T06:28:35.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.812+0000 7f1d567db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5013dd40 con 0x7f1d500b35a0 2023-11-24T06:28:35.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d5013cf10 0x7f1d5013ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d5013cf10 0x7f1d5013ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40232/0 (socket says 172.21.15.64:40232) 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 -- 172.21.15.64:0/3239577509 learned_addr learned my addr 172.21.15.64:0/3239577509 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d557d9640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d500b35a0 0x7f1d501316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d55fda640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d5013fcb0 0x7f1d5013d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 -- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d500b35a0 msgr2=0x7f1d501316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d500b35a0 0x7f1d501316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 -- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d5013fcb0 msgr2=0x7f1d5013d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d5013fcb0 0x7f1d5013d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 -- 172.21.15.64:0/3239577509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d48009a70 con 0x7f1d5013cf10 2023-11-24T06:28:35.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.813+0000 7f1d54fd8640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d5013cf10 0x7f1d5013ef00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c3c0 tx=0x7f1d4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.814+0000 7f1d45ffb640 1 -- 172.21.15.64:0/3239577509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40005ce0 con 0x7f1d5013cf10 2023-11-24T06:28:35.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.814+0000 7f1d45ffb640 1 -- 172.21.15.64:0/3239577509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d40005e40 con 0x7f1d5013cf10 2023-11-24T06:28:35.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.814+0000 7f1d45ffb640 1 -- 172.21.15.64:0/3239577509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d400072d0 con 0x7f1d5013cf10 2023-11-24T06:28:35.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.814+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5013e020 con 0x7f1d5013cf10 2023-11-24T06:28:35.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.814+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d501311a0 con 0x7f1d5013cf10 2023-11-24T06:28:35.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.816+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d501313c0 con 0x7f1d5013cf10 2023-11-24T06:28:35.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.817+0000 7f1d45ffb640 1 -- 172.21.15.64:0/3239577509 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d400074f0 con 0x7f1d5013cf10 2023-11-24T06:28:35.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.817+0000 7f1d45ffb640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1d3406ae40 0x7f1d3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.818+0000 7f1d45ffb640 1 -- 172.21.15.64:0/3239577509 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f1d40088ab0 con 0x7f1d5013cf10 2023-11-24T06:28:35.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.818+0000 7f1d557d9640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1d3406ae40 0x7f1d3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.819+0000 7f1d557d9640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1d3406ae40 0x7f1d3406d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0011e0 tx=0x7f1d4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:35.822+0000 7f1d45ffb640 1 -- 172.21.15.64:0/3239577509 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d40053c40 con 0x7f1d5013cf10 2023-11-24T06:28:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.074+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d50067450 con 0x7f1d5013cf10 2023-11-24T06:28:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.075+0000 7f1d45ffb640 1 -- 172.21.15.64:0/3239577509 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d400170a0 con 0x7f1d5013cf10 2023-11-24T06:28:36.077 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1d3406ae40 msgr2=0x7f1d3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1d3406ae40 0x7f1d3406d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0011e0 tx=0x7f1d4c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d5013cf10 msgr2=0x7f1d5013ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d5013cf10 0x7f1d5013ef00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c3c0 tx=0x7f1d4000c880 comp rx=0 tx=0).stop 2023-11-24T06:28:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 shutdown_connections 2023-11-24T06:28:36.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1d3406ae40 0x7f1d3406d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d5013fcb0 0x7f1d5013d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d5013cf10 0x7f1d5013ef00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 --2- 172.21.15.64:0/3239577509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d500b35a0 0x7f1d501316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 >> 172.21.15.64:0/3239577509 conn(0x7f1d5006a7d0 msgr2=0x7f1d500aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 shutdown_connections 2023-11-24T06:28:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.077+0000 7f1d567db640 1 -- 172.21.15.64:0/3239577509 wait complete. 2023-11-24T06:28:36.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.304+0000 7f6ef5cc2640 1 Processor -- start 2023-11-24T06:28:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.304+0000 7f6ef5cc2640 1 -- start start 2023-11-24T06:28:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.306+0000 7f6ef5cc2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 0x7f6ef00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.306+0000 7f6ef5cc2640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef00b6750 con 0x7f6ef015d590 2023-11-24T06:28:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.306+0000 7f6ef5cc2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef00cf1f0 con 0x7f6ef00ce6f0 2023-11-24T06:28:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.306+0000 7f6ef5cc2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00cef10 con 0x7f6ef00d0280 2023-11-24T06:28:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.306+0000 7f6eef7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6ef015d590 0x7f6ef00cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41626/0 (socket says 172.21.15.64:41626) 2023-11-24T06:28:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.306+0000 7f6eef7fe640 1 -- 172.21.15.64:0/1734339736 learned_addr learned my addr 172.21.15.64:0/1734339736 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 0x7f6ef00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890520918 0 0) 0x7f6ef00b6750 con 0x7f6ef015d590 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed8003520 con 0x7f6ef015d590 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234137536 0 0) 0x7f6ed8003520 con 0x7f6ef015d590 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 msgr2=0x7f6ef00ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 0x7f6ef00ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 >> v1:172.21.15.162:6789/0 conn(0x7f6ef00ce6f0 legacy=0x7f6ef00b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef00b6100 con 0x7f6ef015d590 2023-11-24T06:28:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f6ee0001670 con 0x7f6ef015d590 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 >> v1:172.21.15.64:6789/0 conn(0x7f6ef015d590 legacy=0x7f6ef00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.307+0000 7f6eee7fc640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 0x7f6ed80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eee7fc640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ed80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eee7fc640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ed8004d00 0x7f6ed8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00cef10 con 0x7f6ed8003f20 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed800ba60 con 0x7f6ed8004d00 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed800bc20 con 0x7f6ef00d0280 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ed80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eef7fe640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 0x7f6ed80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eeffff640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ed8004d00 0x7f6ed8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eeeffd640 1 -- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 msgr2=0x7f6ed80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:36.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef00d0280 0x7f6ed80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eeeffd640 1 -- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ed8004d00 msgr2=0x7f6ed8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:36.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ed8004d00 0x7f6ed8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.308+0000 7f6eeeffd640 1 -- 172.21.15.64:0/1734339736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8003520 con 0x7f6ed8003f20 2023-11-24T06:28:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.310+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ed80047c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009ac0 tx=0x7f6ed80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b612b320b314727e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.310+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee402c6b0 con 0x7f6ed8003f20 2023-11-24T06:28:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.310+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee402c810 con 0x7f6ed8003f20 2023-11-24T06:28:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.310+0000 7f6eee7fc640 1 -- 172.21.15.64:0/1734339736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee402cba0 con 0x7f6ed8003f20 2023-11-24T06:28:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.311+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1734339736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 msgr2=0x7f6ed80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.311+0000 7f6ef5cc2640 1 --2- 172.21.15.64:0/1734339736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ed80047c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009ac0 tx=0x7f6ed80057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.311+0000 7f6eeffff640 1 -- 172.21.15.64:0/1734339736 reap_dead start 2023-11-24T06:28:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.311+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1734339736 shutdown_connections 2023-11-24T06:28:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.311+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1734339736 >> 172.21.15.64:0/1734339736 conn(0x7f6ef009c540 msgr2=0x7f6ef00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.311+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1734339736 shutdown_connections 2023-11-24T06:28:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.311+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1734339736 wait complete. 2023-11-24T06:28:36.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.314+0000 7f6ef5cc2640 1 Processor -- start 2023-11-24T06:28:36.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.314+0000 7f6ef5cc2640 1 -- start start 2023-11-24T06:28:36.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6ef5cc2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ef015b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6ef5cc2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ef00ca2e0 0x7f6ef00c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6ef5cc2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef006c230 0x7f6ef006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6ef5cc2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00cb460 con 0x7f6ed8003f20 2023-11-24T06:28:36.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6ef5cc2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef009ddb0 con 0x7f6ef00ca2e0 2023-11-24T06:28:36.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6ef5cc2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef006cc00 con 0x7f6ef006c230 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6eef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ef015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef006c230 0x7f6ef006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef006c230 0x7f6ef006c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44150/0 (socket says 172.21.15.64:44150) 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.315+0000 7f6eeffff640 1 -- 172.21.15.64:0/1083874133 learned_addr learned my addr 172.21.15.64:0/1083874133 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.316+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ef00ca2e0 0x7f6ef00c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.316+0000 7f6eef7fe640 1 -- 172.21.15.64:0/1083874133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef006c230 msgr2=0x7f6ef006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.316+0000 7f6eef7fe640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef006c230 0x7f6ef006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.316+0000 7f6eef7fe640 1 -- 172.21.15.64:0/1083874133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ef00ca2e0 msgr2=0x7f6ef00c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.316+0000 7f6eef7fe640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ef00ca2e0 0x7f6ef00c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.316+0000 7f6eef7fe640 1 -- 172.21.15.64:0/1083874133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef00cb880 con 0x7f6ed8003f20 2023-11-24T06:28:36.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.316+0000 7f6eef7fe640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ef015b090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000eb70 tx=0x7f6ee000c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:36.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.317+0000 7f6ed7fff640 1 -- 172.21.15.64:0/1083874133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000ec60 con 0x7f6ed8003f20 2023-11-24T06:28:36.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.317+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef00ca7e0 con 0x7f6ed8003f20 2023-11-24T06:28:36.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.317+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef00cada0 con 0x7f6ed8003f20 2023-11-24T06:28:36.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.317+0000 7f6ed7fff640 1 -- 172.21.15.64:0/1083874133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee000edc0 con 0x7f6ed8003f20 2023-11-24T06:28:36.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.317+0000 7f6ed7fff640 1 -- 172.21.15.64:0/1083874133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000ce00 con 0x7f6ed8003f20 2023-11-24T06:28:36.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.319+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed0005250 con 0x7f6ed8003f20 2023-11-24T06:28:36.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.320+0000 7f6ed7fff640 1 -- 172.21.15.64:0/1083874133 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ee001d460 con 0x7f6ed8003f20 2023-11-24T06:28:36.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.321+0000 7f6ed7fff640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6edc06ae70 0x7f6edc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.321+0000 7f6ed7fff640 1 -- 172.21.15.64:0/1083874133 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f6ee00893d0 con 0x7f6ed8003f20 2023-11-24T06:28:36.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.322+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6edc06ae70 0x7f6edc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.323+0000 7f6eeeffd640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6edc06ae70 0x7f6edc06d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4036000 tx=0x7f6ee4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:36.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.326+0000 7f6ed7fff640 1 -- 172.21.15.64:0/1083874133 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ee0054560 con 0x7f6ed8003f20 2023-11-24T06:28:36.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.572+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ed0005a80 con 0x7f6ed8003f20 2023-11-24T06:28:36.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.572+0000 7f6ed7fff640 1 -- 172.21.15.64:0/1083874133 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ee0057b80 con 0x7f6ed8003f20 2023-11-24T06:28:36.575 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6edc06ae70 msgr2=0x7f6edc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6edc06ae70 0x7f6edc06d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4036000 tx=0x7f6ee4030000 comp rx=0 tx=0).stop 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 msgr2=0x7f6ef015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ef015b090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000eb70 tx=0x7f6ee000c4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 shutdown_connections 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6edc06ae70 0x7f6edc06d330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef006c230 0x7f6ef006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ef00ca2e0 0x7f6ef00c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 --2- 172.21.15.64:0/1083874133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed8003f20 0x7f6ef015b090 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 >> 172.21.15.64:0/1083874133 conn(0x7f6ef009c540 msgr2=0x7f6ef00b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 shutdown_connections 2023-11-24T06:28:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.576+0000 7f6ef5cc2640 1 -- 172.21.15.64:0/1083874133 wait complete. 2023-11-24T06:28:36.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:36.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.849+0000 7f1949db3640 1 Processor -- start 2023-11-24T06:28:36.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.849+0000 7f1949db3640 1 -- start start 2023-11-24T06:28:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.851+0000 7f1949db3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f194413ff20 0x7f194415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.851+0000 7f1949db3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1944140300 0x7f194406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.851+0000 7f1949db3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f194406a920 0x7f19440b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.851+0000 7f1949db3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194406adb0 con 0x7f194406a920 2023-11-24T06:28:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.851+0000 7f1949db3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19440b5440 con 0x7f194413ff20 2023-11-24T06:28:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.851+0000 7f1949db3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1944161c80 con 0x7f1944140300 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f194406a920 0x7f19440b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f194406a920 0x7f19440b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40266/0 (socket says 172.21.15.64:40266) 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 -- 172.21.15.64:0/4163153133 learned_addr learned my addr 172.21.15.64:0/4163153133 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f1943fff640 1 --2- 172.21.15.64:0/4163153133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1944140300 0x7f194406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f1948db1640 1 --2- 172.21.15.64:0/4163153133 >> v2:172.21.15.162:3300/0 conn(0x7f194413ff20 0x7f194415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 -- 172.21.15.64:0/4163153133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1944140300 msgr2=0x7f194406a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 --2- 172.21.15.64:0/4163153133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1944140300 0x7f194406a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 -- 172.21.15.64:0/4163153133 >> v2:172.21.15.162:3300/0 conn(0x7f194413ff20 msgr2=0x7f194415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 --2- 172.21.15.64:0/4163153133 >> v2:172.21.15.162:3300/0 conn(0x7f194413ff20 0x7f194415e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.852+0000 7f19495b2640 1 -- 172.21.15.64:0/4163153133 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194415e6f0 con 0x7f194406a920 2023-11-24T06:28:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.853+0000 7f19495b2640 1 --2- 172.21.15.64:0/4163153133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194406a920 0x7f19440b3210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19440726c0 tx=0x7f193c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22878efd595b72d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.854+0000 7f19437fe640 1 -- 172.21.15.64:0/4163153133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c02c9f0 con 0x7f194406a920 2023-11-24T06:28:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.854+0000 7f19437fe640 1 -- 172.21.15.64:0/4163153133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c02cb50 con 0x7f194406a920 2023-11-24T06:28:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.854+0000 7f19437fe640 1 -- 172.21.15.64:0/4163153133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0343f0 con 0x7f194406a920 2023-11-24T06:28:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.854+0000 7f1949db3640 1 -- 172.21.15.64:0/4163153133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194406a920 msgr2=0x7f19440b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.854+0000 7f1949db3640 1 --2- 172.21.15.64:0/4163153133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194406a920 0x7f19440b3210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19440726c0 tx=0x7f193c02be50 comp rx=0 tx=0).stop 2023-11-24T06:28:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.855+0000 7f1949db3640 1 -- 172.21.15.64:0/4163153133 shutdown_connections 2023-11-24T06:28:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.855+0000 7f1949db3640 1 --2- 172.21.15.64:0/4163153133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194406a920 0x7f19440b3210 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.855+0000 7f1949db3640 1 --2- 172.21.15.64:0/4163153133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1944140300 0x7f194406a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.855+0000 7f1949db3640 1 --2- 172.21.15.64:0/4163153133 >> v2:172.21.15.162:3300/0 conn(0x7f194413ff20 0x7f194415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.855+0000 7f1949db3640 1 -- 172.21.15.64:0/4163153133 >> 172.21.15.64:0/4163153133 conn(0x7f19440b1cd0 msgr2=0x7f19440b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.855+0000 7f1949db3640 1 -- 172.21.15.64:0/4163153133 shutdown_connections 2023-11-24T06:28:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.855+0000 7f1949db3640 1 -- 172.21.15.64:0/4163153133 wait complete. 2023-11-24T06:28:36.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.857+0000 7f1949db3640 1 Processor -- start 2023-11-24T06:28:36.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.858+0000 7f1949db3640 1 -- start start 2023-11-24T06:28:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.859+0000 7f1949db3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194406a920 0x7f1944157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.859+0000 7f1949db3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194413ff20 0x7f19440ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1949db3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1944140300 0x7f194413c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1949db3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441595b0 con 0x7f194413ff20 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1949db3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19440ae640 con 0x7f1944140300 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1949db3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19440adda0 con 0x7f194406a920 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1948db1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194406a920 0x7f1944157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1943fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194413ff20 0x7f19440ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1948db1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194406a920 0x7f1944157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44162/0 (socket says 172.21.15.64:44162) 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1948db1640 1 -- 172.21.15.64:0/1925254255 learned_addr learned my addr 172.21.15.64:0/1925254255 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f19495b2640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1944140300 0x7f194413c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1943fff640 1 -- 172.21.15.64:0/1925254255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194406a920 msgr2=0x7f1944157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1943fff640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194406a920 0x7f1944157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1943fff640 1 -- 172.21.15.64:0/1925254255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1944140300 msgr2=0x7f194413c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.860+0000 7f1943fff640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1944140300 0x7f194413c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.861+0000 7f1943fff640 1 -- 172.21.15.64:0/1925254255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c009b30 con 0x7f194413ff20 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.861+0000 7f1943fff640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194413ff20 0x7f19440ad4c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f193000e870 tx=0x7f193000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.861+0000 7f19417fa640 1 -- 172.21.15.64:0/1925254255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000ee60 con 0x7f194413ff20 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.861+0000 7f19417fa640 1 -- 172.21.15.64:0/1925254255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193000c940 con 0x7f194413ff20 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.861+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19440ae080 con 0x7f194413ff20 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.861+0000 7f19417fa640 1 -- 172.21.15.64:0/1925254255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000ccd0 con 0x7f194413ff20 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.861+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19441597e0 con 0x7f194413ff20 2023-11-24T06:28:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.863+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1924005250 con 0x7f194413ff20 2023-11-24T06:28:36.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.864+0000 7f19417fa640 1 -- 172.21.15.64:0/1925254255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1930013470 con 0x7f194413ff20 2023-11-24T06:28:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.865+0000 7f19417fa640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192806ae40 0x7f192806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.865+0000 7f19417fa640 1 -- 172.21.15.64:0/1925254255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f19300895f0 con 0x7f194413ff20 2023-11-24T06:28:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.866+0000 7f1948db1640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192806ae40 0x7f192806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.866+0000 7f1948db1640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192806ae40 0x7f192806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1938009c00 tx=0x7f1938009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:36.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:36.873+0000 7f19417fa640 1 -- 172.21.15.64:0/1925254255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1930054650 con 0x7f194413ff20 2023-11-24T06:28:37.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.136+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1924005a80 con 0x7f194413ff20 2023-11-24T06:28:37.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.136+0000 7f19417fa640 1 -- 172.21.15.64:0/1925254255 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1930057c70 con 0x7f194413ff20 2023-11-24T06:28:37.140 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192806ae40 msgr2=0x7f192806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192806ae40 0x7f192806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1938009c00 tx=0x7f1938009190 comp rx=0 tx=0).stop 2023-11-24T06:28:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194413ff20 msgr2=0x7f19440ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194413ff20 0x7f19440ad4c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f193000e870 tx=0x7f193000ed30 comp rx=0 tx=0).stop 2023-11-24T06:28:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 shutdown_connections 2023-11-24T06:28:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192806ae40 0x7f192806d300 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1938009c00 tx=0x7f1938009190 comp rx=0 tx=0).stop 2023-11-24T06:28:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1944140300 0x7f194413c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f194413ff20 0x7f19440ad4c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 --2- 172.21.15.64:0/1925254255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194406a920 0x7f1944157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.139+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 >> 172.21.15.64:0/1925254255 conn(0x7f19440b1cd0 msgr2=0x7f19440cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.140+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 shutdown_connections 2023-11-24T06:28:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.140+0000 7f1949db3640 1 -- 172.21.15.64:0/1925254255 wait complete. 2023-11-24T06:28:37.162 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:28:37.162 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:37.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:37.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.409+0000 7f9d22f5c640 1 Processor -- start 2023-11-24T06:28:37.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.410+0000 7f9d22f5c640 1 -- start start 2023-11-24T06:28:37.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.411+0000 7f9d22f5c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9d1c15d2d0 0x7f9d1c1617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d22f5c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9d1c0cf910 0x7f9d1c071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d22f5c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c15d6b0 con 0x7f9d1c0cf4b0 2023-11-24T06:28:37.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d22f5c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c161d30 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d22f5c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0ce1f0 con 0x7f9d1c0cf910 2023-11-24T06:28:37.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d21f5a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9d1c15d2d0 0x7f9d1c1617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d21759640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9d1c0cf4b0 0x7f9d1c0717b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:49450/0 (socket says 172.21.15.64:49450) 2023-11-24T06:28:37.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d21759640 1 -- 172.21.15.64:0/1005745058 learned_addr learned my addr 172.21.15.64:0/1005745058 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:37.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d2275b640 1 --2- 172.21.15.64:0/1005745058 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c0cf910 0x7f9d1c071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.412+0000 7f9d21f5a640 1 -- 172.21.15.64:0/1005745058 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c0cf910 msgr2=0x7f9d1c071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:37.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.413+0000 7f9d21f5a640 1 --2- 172.21.15.64:0/1005745058 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c0cf910 0x7f9d1c071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.413+0000 7f9d21f5a640 1 -- 172.21.15.64:0/1005745058 >> v1:172.21.15.162:6789/0 conn(0x7f9d1c0cf4b0 legacy=0x7f9d1c0717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.413+0000 7f9d21f5a640 1 -- 172.21.15.64:0/1005745058 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c0ce4d0 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.413+0000 7f9d20f58640 1 -- 172.21.15.64:0/1005745058 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086901868 0 0) 0x7f9d1c15d6b0 con 0x7f9d1c0cf4b0 2023-11-24T06:28:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.414+0000 7f9d21f5a640 1 --2- 172.21.15.64:0/1005745058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c1617f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0ae330 tx=0x7f9d1802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf2fbdd0b209bdb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.414+0000 7f9d20f58640 1 -- 172.21.15.64:0/1005745058 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18031830 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d20f58640 1 -- 172.21.15.64:0/1005745058 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d18031990 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d20f58640 1 -- 172.21.15.64:0/1005745058 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18031d20 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d22f5c640 1 -- 172.21.15.64:0/1005745058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c15d2d0 msgr2=0x7f9d1c1617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/1005745058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c1617f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0ae330 tx=0x7f9d1802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d22f5c640 1 -- 172.21.15.64:0/1005745058 shutdown_connections 2023-11-24T06:28:37.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/1005745058 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c0cf910 0x7f9d1c071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/1005745058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c1617f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.415+0000 7f9d22f5c640 1 -- 172.21.15.64:0/1005745058 >> 172.21.15.64:0/1005745058 conn(0x7f9d1c06a800 msgr2=0x7f9d1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:37.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.416+0000 7f9d22f5c640 1 -- 172.21.15.64:0/1005745058 shutdown_connections 2023-11-24T06:28:37.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.416+0000 7f9d22f5c640 1 -- 172.21.15.64:0/1005745058 wait complete. 2023-11-24T06:28:37.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.418+0000 7f9d22f5c640 1 Processor -- start 2023-11-24T06:28:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.418+0000 7f9d22f5c640 1 -- start start 2023-11-24T06:28:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d22f5c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c0cf4b0 0x7f9d1c144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d22f5c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0cf910 0x7f9d1c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d22f5c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d22f5c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c154f80 con 0x7f9d1c0cf4b0 2023-11-24T06:28:37.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d22f5c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c144b00 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d22f5c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c09ba30 con 0x7f9d1c0cf910 2023-11-24T06:28:37.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d21f5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c0cf4b0 0x7f9d1c144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.419+0000 7f9d2275b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d2275b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:54826/0 (socket says 172.21.15.64:54826) 2023-11-24T06:28:37.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d2275b640 1 -- 172.21.15.64:0/2203463036 learned_addr learned my addr 172.21.15.64:0/2203463036 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:37.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d21759640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0cf910 0x7f9d1c0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d2275b640 1 -- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0cf910 msgr2=0x7f9d1c0aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d2275b640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0cf910 0x7f9d1c0aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d2275b640 1 -- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c0cf4b0 msgr2=0x7f9d1c144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d2275b640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c0cf4b0 0x7f9d1c144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d2275b640 1 -- 172.21.15.64:0/2203463036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18009400 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.420+0000 7f9d21f5a640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c0cf4b0 0x7f9d1c144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.421+0000 7f9d2275b640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c0af190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d14007e90 tx=0x7f9d1400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.421+0000 7f9d127fc640 1 -- 172.21.15.64:0/2203463036 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14014420 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.421+0000 7f9d127fc640 1 -- 172.21.15.64:0/2203463036 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14014580 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.421+0000 7f9d127fc640 1 -- 172.21.15.64:0/2203463036 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14014930 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.421+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c09bd10 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.421+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c09c2d0 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.422+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c09cca0 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.424+0000 7f9d127fc640 1 -- 172.21.15.64:0/2203463036 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d14014a90 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.425+0000 7f9d127fc640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d0006ae40 0x7f9d0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.426+0000 7f9d127fc640 1 -- 172.21.15.64:0/2203463036 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f9d14089df0 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.426+0000 7f9d21f5a640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d0006ae40 0x7f9d0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.427+0000 7f9d21f5a640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d0006ae40 0x7f9d0006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d18036000 tx=0x7f9d1803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.434+0000 7f9d127fc640 1 -- 172.21.15.64:0/2203463036 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d14054ee0 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.721+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d1c067450 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.723+0000 7f9d127fc640 1 -- 172.21.15.64:0/2203463036 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f9d14058500 con 0x7f9d1c15d2d0 2023-11-24T06:28:37.727 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:37.727 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d0006ae40 msgr2=0x7f9d0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d0006ae40 0x7f9d0006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d18036000 tx=0x7f9d1803c040 comp rx=0 tx=0).stop 2023-11-24T06:28:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c15d2d0 msgr2=0x7f9d1c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c0af190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d14007e90 tx=0x7f9d1400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 shutdown_connections 2023-11-24T06:28:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9d0006ae40 0x7f9d0006d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c15d2d0 0x7f9d1c0af190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0cf910 0x7f9d1c0aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 --2- 172.21.15.64:0/2203463036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d1c0cf4b0 0x7f9d1c144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 >> 172.21.15.64:0/2203463036 conn(0x7f9d1c06a800 msgr2=0x7f9d1c06ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.726+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 shutdown_connections 2023-11-24T06:28:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.727+0000 7f9d22f5c640 1 -- 172.21.15.64:0/2203463036 wait complete. 2023-11-24T06:28:37.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:37.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:37.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.945+0000 7f08ab894640 1 Processor -- start 2023-11-24T06:28:37.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.946+0000 7f08ab894640 1 -- start start 2023-11-24T06:28:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08ab894640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08a409a170 0x7f08a40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08ab894640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f08a406c6b0 0x7f08a40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08ab894640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a406cea0 con 0x7f08a40cd240 2023-11-24T06:28:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08ab894640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a406cbc0 con 0x7f08a406c6b0 2023-11-24T06:28:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08ab894640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a406f350 con 0x7f08a409a170 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08a9630640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f08a40cd240 0x7f08a4099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41634/0 (socket says 172.21.15.64:41634) 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08a9630640 1 -- 172.21.15.64:0/659092602 learned_addr learned my addr 172.21.15.64:0/659092602 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08a9e31640 1 --2- 172.21.15.64:0/659092602 >> v2:172.21.15.64:3300/0 conn(0x7f08a406c6b0 0x7f08a40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.949+0000 7f08a8e2f640 1 --2- 172.21.15.64:0/659092602 >> v2:172.21.15.162:3300/0 conn(0x7f08a409a170 0x7f08a40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.950+0000 7f08a9e31640 1 -- 172.21.15.64:0/659092602 >> v2:172.21.15.162:3300/0 conn(0x7f08a409a170 msgr2=0x7f08a40b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.950+0000 7f08a9e31640 1 --2- 172.21.15.64:0/659092602 >> v2:172.21.15.162:3300/0 conn(0x7f08a409a170 0x7f08a40b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.950+0000 7f08a9e31640 1 -- 172.21.15.64:0/659092602 >> v1:172.21.15.64:6789/0 conn(0x7f08a40cd240 legacy=0x7f08a4099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.950+0000 7f08a9e31640 1 -- 172.21.15.64:0/659092602 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a406f630 con 0x7f08a406c6b0 2023-11-24T06:28:37.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.952+0000 7f08a9e31640 1 --2- 172.21.15.64:0/659092602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a406c6b0 0x7f08a40cf640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08a4071800 tx=0x7f089c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ae1a8ae7eb409e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.952+0000 7f089bfff640 1 -- 172.21.15.64:0/659092602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c02cc80 con 0x7f08a406c6b0 2023-11-24T06:28:37.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.952+0000 7f089bfff640 1 -- 172.21.15.64:0/659092602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c02cde0 con 0x7f08a406c6b0 2023-11-24T06:28:37.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.952+0000 7f089bfff640 1 -- 172.21.15.64:0/659092602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c034660 con 0x7f08a406c6b0 2023-11-24T06:28:37.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 -- 172.21.15.64:0/659092602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a406c6b0 msgr2=0x7f08a40cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 --2- 172.21.15.64:0/659092602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a406c6b0 0x7f08a40cf640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08a4071800 tx=0x7f089c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 -- 172.21.15.64:0/659092602 shutdown_connections 2023-11-24T06:28:37.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 --2- 172.21.15.64:0/659092602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a406c6b0 0x7f08a40cf640 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 --2- 172.21.15.64:0/659092602 >> v2:172.21.15.162:3300/0 conn(0x7f08a409a170 0x7f08a40b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 -- 172.21.15.64:0/659092602 >> 172.21.15.64:0/659092602 conn(0x7f08a40b2d00 msgr2=0x7f08a40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:37.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 -- 172.21.15.64:0/659092602 shutdown_connections 2023-11-24T06:28:37.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.953+0000 7f08ab894640 1 -- 172.21.15.64:0/659092602 wait complete. 2023-11-24T06:28:37.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.956+0000 7f08ab894640 1 Processor -- start 2023-11-24T06:28:37.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.956+0000 7f08ab894640 1 -- start start 2023-11-24T06:28:37.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.957+0000 7f08ab894640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a406c6b0 0x7f08a40cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.957+0000 7f08ab894640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a409a170 0x7f08a40c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.957+0000 7f08ab894640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a40cd240 0x7f08a414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.957+0000 7f08ab894640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cadc0 con 0x7f08a409a170 2023-11-24T06:28:37.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.957+0000 7f08ab894640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40ca1f0 con 0x7f08a406c6b0 2023-11-24T06:28:37.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.957+0000 7f08ab894640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40ca490 con 0x7f08a40cd240 2023-11-24T06:28:37.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a409a170 0x7f08a40c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a409a170 0x7f08a40c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40314/0 (socket says 172.21.15.64:40314) 2023-11-24T06:28:37.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 -- 172.21.15.64:0/2437324103 learned_addr learned my addr 172.21.15.64:0/2437324103 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:37.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a9630640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a406c6b0 0x7f08a40cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a9e31640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a40cd240 0x7f08a414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 -- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a40cd240 msgr2=0x7f08a414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a40cd240 0x7f08a414efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 -- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a406c6b0 msgr2=0x7f08a40cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a406c6b0 0x7f08a40cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.958+0000 7f08a8e2f640 1 -- 172.21.15.64:0/2437324103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c009a70 con 0x7f08a409a170 2023-11-24T06:28:37.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.959+0000 7f08a8e2f640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a409a170 0x7f08a40c84a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f089400e9b0 tx=0x7f089400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.959+0000 7f0899ffb640 1 -- 172.21.15.64:0/2437324103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894009670 con 0x7f08a409a170 2023-11-24T06:28:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.959+0000 7f0899ffb640 1 -- 172.21.15.64:0/2437324103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089400c9a0 con 0x7f08a409a170 2023-11-24T06:28:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.959+0000 7f0899ffb640 1 -- 172.21.15.64:0/2437324103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089400cd50 con 0x7f08a409a170 2023-11-24T06:28:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.959+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a40ca770 con 0x7f08a409a170 2023-11-24T06:28:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.959+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a40c89e0 con 0x7f08a409a170 2023-11-24T06:28:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.961+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a40c8c30 con 0x7f08a409a170 2023-11-24T06:28:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.962+0000 7f0899ffb640 1 -- 172.21.15.64:0/2437324103 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0894013470 con 0x7f08a409a170 2023-11-24T06:28:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.963+0000 7f0899ffb640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f088806ae90 0x7f088806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.963+0000 7f0899ffb640 1 -- 172.21.15.64:0/2437324103 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f0894089760 con 0x7f08a409a170 2023-11-24T06:28:37.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.963+0000 7f08a9630640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f088806ae90 0x7f088806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.964+0000 7f08a9630640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f088806ae90 0x7f088806d350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f08a0007fd0 tx=0x7f08a0007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:37.971+0000 7f0899ffb640 1 -- 172.21.15.64:0/2437324103 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08940548d0 con 0x7f08a409a170 2023-11-24T06:28:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.269+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08a40675d0 con 0x7f08a409a170 2023-11-24T06:28:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.269+0000 7f0899ffb640 1 -- 172.21.15.64:0/2437324103 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0894057ef0 con 0x7f08a409a170 2023-11-24T06:28:38.273 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f088806ae90 msgr2=0x7f088806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f088806ae90 0x7f088806d350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f08a0007fd0 tx=0x7f08a0007e80 comp rx=0 tx=0).stop 2023-11-24T06:28:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a409a170 msgr2=0x7f08a40c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a409a170 0x7f08a40c84a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f089400e9b0 tx=0x7f089400ee70 comp rx=0 tx=0).stop 2023-11-24T06:28:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 shutdown_connections 2023-11-24T06:28:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f088806ae90 0x7f088806d350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a40cd240 0x7f08a414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08a409a170 0x7f08a40c84a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 --2- 172.21.15.64:0/2437324103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a406c6b0 0x7f08a40cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 >> 172.21.15.64:0/2437324103 conn(0x7f08a40b2d00 msgr2=0x7f08a40cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 shutdown_connections 2023-11-24T06:28:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.273+0000 7f08ab894640 1 -- 172.21.15.64:0/2437324103 wait complete. 2023-11-24T06:28:38.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:38.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.531+0000 7f5faf59e640 1 Processor -- start 2023-11-24T06:28:38.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.532+0000 7f5faf59e640 1 -- start start 2023-11-24T06:28:38.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.533+0000 7f5faf59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d1d0 0x7f5fb00d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.533+0000 7f5faf59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fb00b1f70 0x7f5fb00d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.533+0000 7f5faf59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb00d0090 con 0x7f5fb015d5b0 2023-11-24T06:28:38.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.533+0000 7f5faf59e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb00d02e0 con 0x7f5fb00b1f70 2023-11-24T06:28:38.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.533+0000 7f5faf59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00d05c0 con 0x7f5fb015d1d0 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.533+0000 7f5faed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5fb015d5b0 0x7f5fb00cdb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41648/0 (socket says 172.21.15.64:41648) 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.533+0000 7f5faed9d640 1 -- 172.21.15.64:0/3085507521 learned_addr learned my addr 172.21.15.64:0/3085507521 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.534+0000 7f5fae59c640 1 --2- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d1d0 0x7f5fb00d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.534+0000 7f5fadd9b640 1 --2- 172.21.15.64:0/3085507521 >> v2:172.21.15.162:3300/0 conn(0x7f5fb00b1f70 0x7f5fb00d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.534+0000 7f5fad59a640 1 -- 172.21.15.64:0/3085507521 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029231573 0 0) 0x7f5fb00d0090 con 0x7f5fb015d5b0 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.534+0000 7f5fad59a640 1 -- 172.21.15.64:0/3085507521 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f98003520 con 0x7f5fb015d5b0 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.534+0000 7f5fadd9b640 1 -- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d1d0 msgr2=0x7f5fb00d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.535+0000 7f5fadd9b640 1 --2- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d1d0 0x7f5fb00d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.535+0000 7f5fadd9b640 1 -- 172.21.15.64:0/3085507521 >> v1:172.21.15.64:6789/0 conn(0x7f5fb015d5b0 legacy=0x7f5fb00cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.535+0000 7f5fadd9b640 1 -- 172.21.15.64:0/3085507521 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00d08a0 con 0x7f5fb00b1f70 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.536+0000 7f5fadd9b640 1 --2- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00d1b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097a0 tx=0x7f5fa402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60c59d8398bb7e05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.536+0000 7f5fad59a640 1 -- 172.21.15.64:0/3085507521 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa402d040 con 0x7f5fb00b1f70 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.536+0000 7f5fad59a640 1 -- 172.21.15.64:0/3085507521 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa4031810 con 0x7f5fb00b1f70 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.536+0000 7f5fad59a640 1 -- 172.21.15.64:0/3085507521 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4031ba0 con 0x7f5fb00b1f70 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 -- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb00b1f70 msgr2=0x7f5fb00d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 --2- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00d1b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097a0 tx=0x7f5fa402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 -- 172.21.15.64:0/3085507521 shutdown_connections 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 --2- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00d1b70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 --2- 172.21.15.64:0/3085507521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d1d0 0x7f5fb00d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 -- 172.21.15.64:0/3085507521 >> 172.21.15.64:0/3085507521 conn(0x7f5fb00d3050 msgr2=0x7f5fb00d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 -- 172.21.15.64:0/3085507521 shutdown_connections 2023-11-24T06:28:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.537+0000 7f5faf59e640 1 -- 172.21.15.64:0/3085507521 wait complete. 2023-11-24T06:28:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.539+0000 7f5faf59e640 1 Processor -- start 2023-11-24T06:28:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.539+0000 7f5faf59e640 1 -- start start 2023-11-24T06:28:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5faf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5faf59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb015d1d0 0x7f5fb00b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5faf59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d5b0 0x7f5fb014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5faf59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00b7960 con 0x7f5fb00b1f70 2023-11-24T06:28:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5faf59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00b7c40 con 0x7f5fb015d1d0 2023-11-24T06:28:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5faf59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00b7f20 con 0x7f5fb015d5b0 2023-11-24T06:28:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5fae59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5fae59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00b8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40320/0 (socket says 172.21.15.64:40320) 2023-11-24T06:28:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5fae59c640 1 -- 172.21.15.64:0/62690647 learned_addr learned my addr 172.21.15.64:0/62690647 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.540+0000 7f5faed9d640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d5b0 0x7f5fb014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fadd9b640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb015d1d0 0x7f5fb00b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fae59c640 1 -- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d5b0 msgr2=0x7f5fb014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fae59c640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d5b0 0x7f5fb014ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fae59c640 1 -- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb015d1d0 msgr2=0x7f5fb00b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fae59c640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb015d1d0 0x7f5fb00b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fae59c640 1 -- 172.21.15.64:0/62690647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4009400 con 0x7f5fb00b1f70 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fadd9b640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb015d1d0 0x7f5fb00b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.541+0000 7f5fae59c640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00b8a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9b0 tx=0x7f5fa000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.542+0000 7f5f9effd640 1 -- 172.21.15.64:0/62690647 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0009670 con 0x7f5fb00b1f70 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.542+0000 7f5f9effd640 1 -- 172.21.15.64:0/62690647 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa000c9a0 con 0x7f5fb00b1f70 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.542+0000 7f5f9effd640 1 -- 172.21.15.64:0/62690647 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000cd50 con 0x7f5fb00b1f70 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.542+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb00b8200 con 0x7f5fb00b1f70 2023-11-24T06:28:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.542+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb00b6b90 con 0x7f5fb00b1f70 2023-11-24T06:28:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.543+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f94005250 con 0x7f5fb00b1f70 2023-11-24T06:28:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.544+0000 7f5f9effd640 1 -- 172.21.15.64:0/62690647 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fa0013470 con 0x7f5fb00b1f70 2023-11-24T06:28:38.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.546+0000 7f5f9effd640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f9006ae40 0x7f5f9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.546+0000 7f5f9effd640 1 -- 172.21.15.64:0/62690647 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f5fa0089810 con 0x7f5fb00b1f70 2023-11-24T06:28:38.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.546+0000 7f5fadd9b640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f9006ae40 0x7f5f9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.547+0000 7f5fadd9b640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f9006ae40 0x7f5f9006d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4036000 tx=0x7f5fa403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.550+0000 7f5f9effd640 1 -- 172.21.15.64:0/62690647 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5fa0054910 con 0x7f5fb00b1f70 2023-11-24T06:28:38.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.840+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f94005a80 con 0x7f5fb00b1f70 2023-11-24T06:28:38.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.841+0000 7f5f9effd640 1 -- 172.21.15.64:0/62690647 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fa0017020 con 0x7f5fb00b1f70 2023-11-24T06:28:38.844 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:38.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f9006ae40 msgr2=0x7f5f9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f9006ae40 0x7f5f9006d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4036000 tx=0x7f5fa403c040 comp rx=0 tx=0).stop 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb00b1f70 msgr2=0x7f5fb00b8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00b8a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9b0 tx=0x7f5fa000ee70 comp rx=0 tx=0).stop 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 shutdown_connections 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f9006ae40 0x7f5f9006d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fb015d5b0 0x7f5fb014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fb015d1d0 0x7f5fb00b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 --2- 172.21.15.64:0/62690647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb00b1f70 0x7f5fb00b8a00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 >> 172.21.15.64:0/62690647 conn(0x7f5fb00d3050 msgr2=0x7f5fb015f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 shutdown_connections 2023-11-24T06:28:38.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:38.844+0000 7f5faf59e640 1 -- 172.21.15.64:0/62690647 wait complete. 2023-11-24T06:28:38.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.075+0000 7f6ab1783640 1 Processor -- start 2023-11-24T06:28:39.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.075+0000 7f6ab1783640 1 -- start start 2023-11-24T06:28:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.077+0000 7f6ab1783640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aac0adf20 0x7f6aac070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.077+0000 7f6ab1783640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac15f4e0 con 0x7f6aac15bf10 2023-11-24T06:28:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.077+0000 7f6ab1783640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac071130 con 0x7f6aac0ccb50 2023-11-24T06:28:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.077+0000 7f6ab1783640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac15f970 con 0x7f6aac0adf20 2023-11-24T06:28:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.077+0000 7f6aabfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aac0adf20 0x7f6aac070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.077+0000 7f6aabfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aac0adf20 0x7f6aac070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40328/0 (socket says 172.21.15.64:40328) 2023-11-24T06:28:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.077+0000 7f6aabfff640 1 -- 172.21.15.64:0/893787354 learned_addr learned my addr 172.21.15.64:0/893787354 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.078+0000 7f6aabfff640 1 -- 172.21.15.64:0/893787354 >> v1:172.21.15.162:6789/0 conn(0x7f6aac0ccb50 legacy=0x7f6aac143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.078+0000 7f6aabfff640 1 -- 172.21.15.64:0/893787354 >> v1:172.21.15.64:6789/0 conn(0x7f6aac15bf10 legacy=0x7f6aac0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.078+0000 7f6aabfff640 1 -- 172.21.15.64:0/893787354 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac15c390 con 0x7f6aac0adf20 2023-11-24T06:28:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.079+0000 7f6aabfff640 1 --2- 172.21.15.64:0/893787354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0adf20 0x7f6aac070a30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009870 tx=0x7f6a9c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac9b10033e43f07 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.079+0000 7f6aab7fe640 1 -- 172.21.15.64:0/893787354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c030860 con 0x7f6aac0adf20 2023-11-24T06:28:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.079+0000 7f6aab7fe640 1 -- 172.21.15.64:0/893787354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c0309c0 con 0x7f6aac0adf20 2023-11-24T06:28:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.079+0000 7f6aab7fe640 1 -- 172.21.15.64:0/893787354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c030d50 con 0x7f6aac0adf20 2023-11-24T06:28:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.080+0000 7f6ab1783640 1 -- 172.21.15.64:0/893787354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0adf20 msgr2=0x7f6aac070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.080+0000 7f6ab1783640 1 --2- 172.21.15.64:0/893787354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0adf20 0x7f6aac070a30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009870 tx=0x7f6a9c02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.080+0000 7f6ab1783640 1 -- 172.21.15.64:0/893787354 shutdown_connections 2023-11-24T06:28:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.080+0000 7f6ab1783640 1 --2- 172.21.15.64:0/893787354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0adf20 0x7f6aac070a30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.080+0000 7f6ab1783640 1 -- 172.21.15.64:0/893787354 >> 172.21.15.64:0/893787354 conn(0x7f6aac06a7d0 msgr2=0x7f6aac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.080+0000 7f6ab1783640 1 -- 172.21.15.64:0/893787354 shutdown_connections 2023-11-24T06:28:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.080+0000 7f6ab1783640 1 -- 172.21.15.64:0/893787354 wait complete. 2023-11-24T06:28:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.082+0000 7f6ab1783640 1 Processor -- start 2023-11-24T06:28:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.083+0000 7f6ab1783640 1 -- start start 2023-11-24T06:28:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6ab1783640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aac0adf20 0x7f6aac15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6ab1783640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0ccb50 0x7f6aac0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6ab1783640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aac15bf10 0x7f6aac143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6ab1783640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0c3d00 con 0x7f6aac0ccb50 2023-11-24T06:28:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6ab1783640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1f4910 con 0x7f6aac0adf20 2023-11-24T06:28:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6ab1783640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1f4bf0 con 0x7f6aac15bf10 2023-11-24T06:28:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aac0adf20 0x7f6aac15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aac0adf20 0x7f6aac15a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:54854/0 (socket says 172.21.15.64:54854) 2023-11-24T06:28:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6aabfff640 1 -- 172.21.15.64:0/2178711736 learned_addr learned my addr 172.21.15.64:0/2178711736 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.084+0000 7f6aa3fff640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0ccb50 0x7f6aac0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.085+0000 7f6ab0f82640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aac15bf10 0x7f6aac143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.085+0000 7f6aabfff640 1 -- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aac15bf10 msgr2=0x7f6aac143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.085+0000 7f6aabfff640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aac15bf10 0x7f6aac143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.085+0000 7f6aabfff640 1 -- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0ccb50 msgr2=0x7f6aac0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.085+0000 7f6aabfff640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0ccb50 0x7f6aac0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.085+0000 7f6aabfff640 1 -- 172.21.15.64:0/2178711736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0094d0 con 0x7f6aac0adf20 2023-11-24T06:28:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.085+0000 7f6aabfff640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aac0adf20 0x7f6aac15a8a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.086+0000 7f6aa97fa640 1 -- 172.21.15.64:0/2178711736 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c030830 con 0x7f6aac0adf20 2023-11-24T06:28:39.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.086+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac1f4ed0 con 0x7f6aac0adf20 2023-11-24T06:28:39.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.086+0000 7f6aa97fa640 1 -- 172.21.15.64:0/2178711736 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c039d90 con 0x7f6aac0adf20 2023-11-24T06:28:39.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.086+0000 7f6aa97fa640 1 -- 172.21.15.64:0/2178711736 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c043670 con 0x7f6aac0adf20 2023-11-24T06:28:39.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.086+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac15ade0 con 0x7f6aac0adf20 2023-11-24T06:28:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.087+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac067450 con 0x7f6aac0adf20 2023-11-24T06:28:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.088+0000 7f6aa97fa640 1 -- 172.21.15.64:0/2178711736 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a9c0437d0 con 0x7f6aac0adf20 2023-11-24T06:28:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.089+0000 7f6aa97fa640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a8c06ae40 0x7f6a8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.089+0000 7f6aa97fa640 1 -- 172.21.15.64:0/2178711736 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f6a9c0abf20 con 0x7f6aac0adf20 2023-11-24T06:28:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.090+0000 7f6aa3fff640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a8c06ae40 0x7f6a8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.090+0000 7f6aa3fff640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a8c06ae40 0x7f6a8c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6aac15e440 tx=0x7f6a94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.100+0000 7f6aa97fa640 1 -- 172.21.15.64:0/2178711736 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a9c077000 con 0x7f6aac0adf20 2023-11-24T06:28:39.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.394+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6aac15e470 con 0x7f6aac0adf20 2023-11-24T06:28:39.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.395+0000 7f6aa97fa640 1 -- 172.21.15.64:0/2178711736 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a9c034020 con 0x7f6aac0adf20 2023-11-24T06:28:39.398 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a8c06ae40 msgr2=0x7f6a8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a8c06ae40 0x7f6a8c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6aac15e440 tx=0x7f6a94009040 comp rx=0 tx=0).stop 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aac0adf20 msgr2=0x7f6aac15a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aac0adf20 0x7f6aac15a8a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c004730 comp rx=0 tx=0).stop 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 shutdown_connections 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a8c06ae40 0x7f6a8c06d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aac15bf10 0x7f6aac143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aac0ccb50 0x7f6aac0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 --2- 172.21.15.64:0/2178711736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aac0adf20 0x7f6aac15a8a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 >> 172.21.15.64:0/2178711736 conn(0x7f6aac06a7d0 msgr2=0x7f6aac15cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 shutdown_connections 2023-11-24T06:28:39.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.398+0000 7f6ab1783640 1 -- 172.21.15.64:0/2178711736 wait complete. 2023-11-24T06:28:39.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:39.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.640+0000 7fd5fbfff640 1 Processor -- start 2023-11-24T06:28:39.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.640+0000 7fd5fbfff640 1 -- start start 2023-11-24T06:28:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.643+0000 7fd5fbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5fc143c40 0x7fd5fc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.643+0000 7fd5fbfff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd5fc0ccc10 0x7fd5fc06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.643+0000 7fd5fbfff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc070e50 con 0x7fd5fc0717d0 2023-11-24T06:28:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.643+0000 7fd5fbfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0723c0 con 0x7fd5fc143c40 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.643+0000 7fd5fbfff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc144020 con 0x7fd5fc0ccc10 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5faffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5fc143c40 0x7fd5fc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5faffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5fc143c40 0x7fd5fc071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40334/0 (socket says 172.21.15.64:40334) 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5faffd640 1 -- 172.21.15.64:0/1195393040 learned_addr learned my addr 172.21.15.64:0/1195393040 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5fb7fe640 1 --2- 172.21.15.64:0/1195393040 >> v2:172.21.15.162:3300/0 conn(0x7fd5fc0ccc10 0x7fd5fc06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5faffd640 1 -- 172.21.15.64:0/1195393040 >> v2:172.21.15.162:3300/0 conn(0x7fd5fc0ccc10 msgr2=0x7fd5fc06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5faffd640 1 --2- 172.21.15.64:0/1195393040 >> v2:172.21.15.162:3300/0 conn(0x7fd5fc0ccc10 0x7fd5fc06d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5faffd640 1 -- 172.21.15.64:0/1195393040 >> v1:172.21.15.162:6789/0 conn(0x7fd5fc0717d0 legacy=0x7fd5fc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.644+0000 7fd5faffd640 1 -- 172.21.15.64:0/1195393040 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc071e00 con 0x7fd5fc143c40 2023-11-24T06:28:39.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.645+0000 7fd5faffd640 1 --2- 172.21.15.64:0/1195393040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc143c40 0x7fd5fc071290 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc070690 tx=0x7fd5ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d2c7fb556d969 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.646+0000 7fd5f9ffb640 1 -- 172.21.15.64:0/1195393040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec031870 con 0x7fd5fc143c40 2023-11-24T06:28:39.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.646+0000 7fd5f9ffb640 1 -- 172.21.15.64:0/1195393040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec0319d0 con 0x7fd5fc143c40 2023-11-24T06:28:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.646+0000 7fd5f9ffb640 1 -- 172.21.15.64:0/1195393040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec031d60 con 0x7fd5fc143c40 2023-11-24T06:28:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 -- 172.21.15.64:0/1195393040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc143c40 msgr2=0x7fd5fc071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/1195393040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc143c40 0x7fd5fc071290 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc070690 tx=0x7fd5ec02b910 comp rx=0 tx=0).stop 2023-11-24T06:28:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 -- 172.21.15.64:0/1195393040 shutdown_connections 2023-11-24T06:28:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/1195393040 >> v2:172.21.15.162:3300/0 conn(0x7fd5fc0ccc10 0x7fd5fc06d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/1195393040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc143c40 0x7fd5fc071290 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 -- 172.21.15.64:0/1195393040 >> 172.21.15.64:0/1195393040 conn(0x7fd5fc06a800 msgr2=0x7fd5fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 -- 172.21.15.64:0/1195393040 shutdown_connections 2023-11-24T06:28:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.647+0000 7fd5fbfff640 1 -- 172.21.15.64:0/1195393040 wait complete. 2023-11-24T06:28:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.649+0000 7fd5fbfff640 1 Processor -- start 2023-11-24T06:28:39.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.649+0000 7fd5fbfff640 1 -- start start 2023-11-24T06:28:39.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc0717d0 0x7fd5fc0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fbfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5fc0ccc10 0x7fd5fc0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fbfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5fc143c40 0x7fd5fc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fbfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cb590 con 0x7fd5fc0717d0 2023-11-24T06:28:39.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fbfff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0ca290 con 0x7fd5fc0ccc10 2023-11-24T06:28:39.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fbfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0ca570 con 0x7fd5fc143c40 2023-11-24T06:28:39.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc0717d0 0x7fd5fc0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc0717d0 0x7fd5fc0cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40348/0 (socket says 172.21.15.64:40348) 2023-11-24T06:28:39.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 -- 172.21.15.64:0/4034474345 learned_addr learned my addr 172.21.15.64:0/4034474345 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fa7fc640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5fc0ccc10 0x7fd5fc0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5fb7fe640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5fc143c40 0x7fd5fc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 -- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5fc143c40 msgr2=0x7fd5fc0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5fc143c40 0x7fd5fc0afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 -- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5fc0ccc10 msgr2=0x7fd5fc0c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5fc0ccc10 0x7fd5fc0c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.651+0000 7fd5faffd640 1 -- 172.21.15.64:0/4034474345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0094d0 con 0x7fd5fc0717d0 2023-11-24T06:28:39.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.652+0000 7fd5faffd640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc0717d0 0x7fd5fc0cb050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec000c00 tx=0x7fd5ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.652+0000 7fd5e37fe640 1 -- 172.21.15.64:0/4034474345 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec03aca0 con 0x7fd5fc0717d0 2023-11-24T06:28:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.652+0000 7fd5e37fe640 1 -- 172.21.15.64:0/4034474345 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec03ae00 con 0x7fd5fc0717d0 2023-11-24T06:28:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.652+0000 7fd5e37fe640 1 -- 172.21.15.64:0/4034474345 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec044630 con 0x7fd5fc0717d0 2023-11-24T06:28:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.652+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc0ca850 con 0x7fd5fc0717d0 2023-11-24T06:28:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.652+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0c9f40 con 0x7fd5fc0717d0 2023-11-24T06:28:39.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.654+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc0cad70 con 0x7fd5fc0717d0 2023-11-24T06:28:39.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.655+0000 7fd5e37fe640 1 -- 172.21.15.64:0/4034474345 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5ec04d450 con 0x7fd5fc0717d0 2023-11-24T06:28:39.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.656+0000 7fd5e37fe640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e806ae40 0x7fd5e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.656+0000 7fd5e37fe640 1 -- 172.21.15.64:0/4034474345 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fd5ec02d040 con 0x7fd5fc0717d0 2023-11-24T06:28:39.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.657+0000 7fd5fa7fc640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e806ae40 0x7fd5e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.658+0000 7fd5fa7fc640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e806ae40 0x7fd5e806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0c5180 tx=0x7fd5f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.661+0000 7fd5e37fe640 1 -- 172.21.15.64:0/4034474345 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5fc0cad70 con 0x7fd5fc0717d0 2023-11-24T06:28:39.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.974+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd5fc0cad70 con 0x7fd5fc0717d0 2023-11-24T06:28:39.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.975+0000 7fd5e37fe640 1 -- 172.21.15.64:0/4034474345 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd5ec076bc0 con 0x7fd5fc0717d0 2023-11-24T06:28:39.979 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:39.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.978+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e806ae40 msgr2=0x7fd5e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.978+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e806ae40 0x7fd5e806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0c5180 tx=0x7fd5f0009040 comp rx=0 tx=0).stop 2023-11-24T06:28:39.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.978+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc0717d0 msgr2=0x7fd5fc0cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.978+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc0717d0 0x7fd5fc0cb050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec000c00 tx=0x7fd5ec0046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 shutdown_connections 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e806ae40 0x7fd5e806d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5fc143c40 0x7fd5fc0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5fc0ccc10 0x7fd5fc0c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 --2- 172.21.15.64:0/4034474345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5fc0717d0 0x7fd5fc0cb050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 >> 172.21.15.64:0/4034474345 conn(0x7fd5fc06a800 msgr2=0x7fd5fc0aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 shutdown_connections 2023-11-24T06:28:39.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:39.979+0000 7fd5fbfff640 1 -- 172.21.15.64:0/4034474345 wait complete. 2023-11-24T06:28:40.000 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2023-11-24T06:28:40.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:40.001 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:40.079 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:28:40.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:40.079 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:28:40.136 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:28:40.136 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:40.183 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:28:40.183 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:40.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:40.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.533+0000 7fbcb33c1640 1 Processor -- start 2023-11-24T06:28:40.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.533+0000 7fbcb33c1640 1 -- start start 2023-11-24T06:28:40.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb33c1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcac13cfa0 0x7fbcac15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb33c1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13fcb0 0x7fbcac0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb33c1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac0cdec0 con 0x7fbcac0cd530 2023-11-24T06:28:40.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb33c1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac0ce1a0 con 0x7fbcac13cfa0 2023-11-24T06:28:40.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb33c1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0ce480 con 0x7fbcac13fcb0 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb2bc0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13fcb0 0x7fbcac0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb1bbe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbcac0cd530 0x7fbcac13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:49486/0 (socket says 172.21.15.64:49486) 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.535+0000 7fbcb1bbe640 1 -- 172.21.15.64:0/3299083341 learned_addr learned my addr 172.21.15.64:0/3299083341 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb23bf640 1 --2- 172.21.15.64:0/3299083341 >> v2:172.21.15.162:3300/0 conn(0x7fbcac13cfa0 0x7fbcac15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb13bd640 1 -- 172.21.15.64:0/3299083341 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466899362 0 0) 0x7fbcac0cdec0 con 0x7fbcac0cd530 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb13bd640 1 -- 172.21.15.64:0/3299083341 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc98005730 con 0x7fbcac0cd530 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb2bc0640 1 -- 172.21.15.64:0/3299083341 >> v2:172.21.15.162:3300/0 conn(0x7fbcac13cfa0 msgr2=0x7fbcac15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:40.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb2bc0640 1 --2- 172.21.15.64:0/3299083341 >> v2:172.21.15.162:3300/0 conn(0x7fbcac13cfa0 0x7fbcac15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb2bc0640 1 -- 172.21.15.64:0/3299083341 >> v1:172.21.15.162:6789/0 conn(0x7fbcac0cd530 legacy=0x7fbcac13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb2bc0640 1 -- 172.21.15.64:0/3299083341 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0ce760 con 0x7fbcac13fcb0 2023-11-24T06:28:40.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.536+0000 7fbcb23bf640 1 --2- 172.21.15.64:0/3299083341 >> v2:172.21.15.162:3300/0 conn(0x7fbcac13cfa0 0x7fbcac15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:40.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.538+0000 7fbcb2bc0640 1 --2- 172.21.15.64:0/3299083341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13fcb0 0x7fbcac0b2040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009d90 tx=0x7fbca402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d53796b014310a12 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.538+0000 7fbcb13bd640 1 -- 172.21.15.64:0/3299083341 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca402cbc0 con 0x7fbcac13fcb0 2023-11-24T06:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.538+0000 7fbcb13bd640 1 -- 172.21.15.64:0/3299083341 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca402cd20 con 0x7fbcac13fcb0 2023-11-24T06:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.538+0000 7fbcb13bd640 1 -- 172.21.15.64:0/3299083341 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4034660 con 0x7fbcac13fcb0 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 -- 172.21.15.64:0/3299083341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13fcb0 msgr2=0x7fbcac0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/3299083341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13fcb0 0x7fbcac0b2040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009d90 tx=0x7fbca402be10 comp rx=0 tx=0).stop 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 -- 172.21.15.64:0/3299083341 shutdown_connections 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/3299083341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13fcb0 0x7fbcac0b2040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/3299083341 >> v2:172.21.15.162:3300/0 conn(0x7fbcac13cfa0 0x7fbcac15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 -- 172.21.15.64:0/3299083341 >> 172.21.15.64:0/3299083341 conn(0x7fbcac06a800 msgr2=0x7fbcac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 -- 172.21.15.64:0/3299083341 shutdown_connections 2023-11-24T06:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.539+0000 7fbcb33c1640 1 -- 172.21.15.64:0/3299083341 wait complete. 2023-11-24T06:28:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.542+0000 7fbcb33c1640 1 Processor -- start 2023-11-24T06:28:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.542+0000 7fbcb33c1640 1 -- start start 2023-11-24T06:28:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb33c1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcac0cd530 0x7fbcac1e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb33c1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13cfa0 0x7fbcac071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb33c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcac13fcb0 0x7fbcac071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb33c1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1f6590 con 0x7fbcac13fcb0 2023-11-24T06:28:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb33c1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1f70d0 con 0x7fbcac0cd530 2023-11-24T06:28:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb33c1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1e9e20 con 0x7fbcac13cfa0 2023-11-24T06:28:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb23bf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcac0cd530 0x7fbcac1e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb23bf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcac0cd530 0x7fbcac1e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:54892/0 (socket says 172.21.15.64:54892) 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb23bf640 1 -- 172.21.15.64:0/1674455491 learned_addr learned my addr 172.21.15.64:0/1674455491 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb2bc0640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcac13fcb0 0x7fbcac071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.544+0000 7fbcb1bbe640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13cfa0 0x7fbcac071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.545+0000 7fbcb23bf640 1 -- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13cfa0 msgr2=0x7fbcac071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.545+0000 7fbcb23bf640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13cfa0 0x7fbcac071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.545+0000 7fbcb23bf640 1 -- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcac13fcb0 msgr2=0x7fbcac071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.545+0000 7fbcb23bf640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcac13fcb0 0x7fbcac071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.545+0000 7fbcb23bf640 1 -- 172.21.15.64:0/1674455491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8009400 con 0x7fbcac0cd530 2023-11-24T06:28:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.545+0000 7fbcb23bf640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcac0cd530 0x7fbcac1e98e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbca800ebe0 tx=0x7fbca800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.545+0000 7fbca2ffd640 1 -- 172.21.15.64:0/1674455491 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8009760 con 0x7fbcac0cd530 2023-11-24T06:28:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.546+0000 7fbca2ffd640 1 -- 172.21.15.64:0/1674455491 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca800cc30 con 0x7fbcac0cd530 2023-11-24T06:28:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.546+0000 7fbca2ffd640 1 -- 172.21.15.64:0/1674455491 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca800eee0 con 0x7fbcac0cd530 2023-11-24T06:28:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.546+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca4009a70 con 0x7fbcac0cd530 2023-11-24T06:28:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.546+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac1f6a80 con 0x7fbcac0cd530 2023-11-24T06:28:40.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.547+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac067450 con 0x7fbcac0cd530 2023-11-24T06:28:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.549+0000 7fbca2ffd640 1 -- 172.21.15.64:0/1674455491 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbca8005990 con 0x7fbcac0cd530 2023-11-24T06:28:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.550+0000 7fbca2ffd640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbc9006ae40 0x7fbc9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.550+0000 7fbca2ffd640 1 -- 172.21.15.64:0/1674455491 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fbca8089e40 con 0x7fbcac0cd530 2023-11-24T06:28:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.550+0000 7fbcb1bbe640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbc9006ae40 0x7fbc9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.551+0000 7fbcb1bbe640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbc9006ae40 0x7fbc9006d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0720d0 tx=0x7fbc9c007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.561+0000 7fbca2ffd640 1 -- 172.21.15.64:0/1674455491 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbca8054f40 con 0x7fbcac0cd530 2023-11-24T06:28:40.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.842+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcac0cbff0 con 0x7fbcac0cd530 2023-11-24T06:28:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.844+0000 7fbca2ffd640 1 -- 172.21.15.64:0/1674455491 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7fbca8021090 con 0x7fbcac0cd530 2023-11-24T06:28:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:40.849 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:40.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbc9006ae40 msgr2=0x7fbc9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbc9006ae40 0x7fbc9006d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0720d0 tx=0x7fbc9c007d30 comp rx=0 tx=0).stop 2023-11-24T06:28:40.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcac0cd530 msgr2=0x7fbcac1e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcac0cd530 0x7fbcac1e98e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbca800ebe0 tx=0x7fbca800c520 comp rx=0 tx=0).stop 2023-11-24T06:28:40.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 shutdown_connections 2023-11-24T06:28:40.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbc9006ae40 0x7fbc9006d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcac13fcb0 0x7fbcac071ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcac13cfa0 0x7fbcac071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 --2- 172.21.15.64:0/1674455491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcac0cd530 0x7fbcac1e98e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 >> 172.21.15.64:0/1674455491 conn(0x7fbcac06a800 msgr2=0x7fbcac0aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:40.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 shutdown_connections 2023-11-24T06:28:40.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:40.847+0000 7fbcb33c1640 1 -- 172.21.15.64:0/1674455491 wait complete. 2023-11-24T06:28:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:40.871 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:41.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.103+0000 7f056cb73640 1 Processor -- start 2023-11-24T06:28:41.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.103+0000 7f056cb73640 1 -- start start 2023-11-24T06:28:41.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.104+0000 7f056cb73640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05680ccb50 0x7f0568070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.104+0000 7f056cb73640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f056815bf10 0x7f0568143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.104+0000 7f056cb73640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056815f4e0 con 0x7f05680adf20 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.104+0000 7f056cb73640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056815f970 con 0x7f056815bf10 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.104+0000 7f056cb73640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0568071130 con 0x7f05680ccb50 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f0567fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f056815bf10 0x7f0568143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f05677fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f05680adf20 0x7f05680cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41666/0 (socket says 172.21.15.64:41666) 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f05677fe640 1 -- 172.21.15.64:0/1371173692 learned_addr learned my addr 172.21.15.64:0/1371173692 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f0566ffd640 1 --2- 172.21.15.64:0/1371173692 >> v2:172.21.15.162:3300/0 conn(0x7f05680ccb50 0x7f0568070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f0567fff640 1 -- 172.21.15.64:0/1371173692 >> v2:172.21.15.162:3300/0 conn(0x7f05680ccb50 msgr2=0x7f0568070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f0567fff640 1 --2- 172.21.15.64:0/1371173692 >> v2:172.21.15.162:3300/0 conn(0x7f05680ccb50 0x7f0568070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f0567fff640 1 -- 172.21.15.64:0/1371173692 >> v1:172.21.15.64:6789/0 conn(0x7f05680adf20 legacy=0x7f05680cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.105+0000 7f0567fff640 1 -- 172.21.15.64:0/1371173692 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056815ebb0 con 0x7f056815bf10 2023-11-24T06:28:41.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.107+0000 7f0567fff640 1 --2- 172.21.15.64:0/1371173692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f0568143d30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f056815e9e0 tx=0x7f056002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38484e4ec4f9ccde server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.107+0000 7f05667fc640 1 -- 172.21.15.64:0/1371173692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056002cbf0 con 0x7f056815bf10 2023-11-24T06:28:41.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f05667fc640 1 -- 172.21.15.64:0/1371173692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056002cd50 con 0x7f056815bf10 2023-11-24T06:28:41.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f05667fc640 1 -- 172.21.15.64:0/1371173692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560034660 con 0x7f056815bf10 2023-11-24T06:28:41.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f056cb73640 1 -- 172.21.15.64:0/1371173692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 msgr2=0x7f0568143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f056cb73640 1 --2- 172.21.15.64:0/1371173692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f0568143d30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f056815e9e0 tx=0x7f056002be40 comp rx=0 tx=0).stop 2023-11-24T06:28:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f056cb73640 1 -- 172.21.15.64:0/1371173692 shutdown_connections 2023-11-24T06:28:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f056cb73640 1 --2- 172.21.15.64:0/1371173692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f0568143d30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f056cb73640 1 --2- 172.21.15.64:0/1371173692 >> v2:172.21.15.162:3300/0 conn(0x7f05680ccb50 0x7f0568070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.108+0000 7f056cb73640 1 -- 172.21.15.64:0/1371173692 >> 172.21.15.64:0/1371173692 conn(0x7f056806a7d0 msgr2=0x7f056806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.109+0000 7f056cb73640 1 -- 172.21.15.64:0/1371173692 shutdown_connections 2023-11-24T06:28:41.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.109+0000 7f056cb73640 1 -- 172.21.15.64:0/1371173692 wait complete. 2023-11-24T06:28:41.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.111+0000 7f056cb73640 1 Processor -- start 2023-11-24T06:28:41.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.111+0000 7f056cb73640 1 -- start start 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f056cb73640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05680adf20 0x7f05680b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f056cb73640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05680ccb50 0x7f05681f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f056cb73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f056815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f056cb73640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681f6910 con 0x7f056815bf10 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f056cb73640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681f6bf0 con 0x7f05680adf20 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f056cb73640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b1f30 con 0x7f05680ccb50 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f0567fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f056815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f0567fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f056815a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40380/0 (socket says 172.21.15.64:40380) 2023-11-24T06:28:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.113+0000 7f0567fff640 1 -- 172.21.15.64:0/3903701389 learned_addr learned my addr 172.21.15.64:0/3903701389 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:41.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.114+0000 7f0566ffd640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05680ccb50 0x7f05681f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.114+0000 7f0567fff640 1 -- 172.21.15.64:0/3903701389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05680ccb50 msgr2=0x7f05681f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.114+0000 7f0567fff640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05680ccb50 0x7f05681f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.114+0000 7f0567fff640 1 -- 172.21.15.64:0/3903701389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05680adf20 msgr2=0x7f05680b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.114+0000 7f0567fff640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05680adf20 0x7f05680b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.114+0000 7f0567fff640 1 -- 172.21.15.64:0/3903701389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009a70 con 0x7f056815bf10 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.114+0000 7f0567fff640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f056815a7e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0560009e80 tx=0x7f0560004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.115+0000 7f054ffff640 1 -- 172.21.15.64:0/3903701389 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560040650 con 0x7f056815bf10 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.115+0000 7f054ffff640 1 -- 172.21.15.64:0/3903701389 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05600407b0 con 0x7f056815bf10 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.115+0000 7f054ffff640 1 -- 172.21.15.64:0/3903701389 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560040b60 con 0x7f056815bf10 2023-11-24T06:28:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.115+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05680b2210 con 0x7f056815bf10 2023-11-24T06:28:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.116+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05681f6ed0 con 0x7f056815bf10 2023-11-24T06:28:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.117+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0568067450 con 0x7f056815bf10 2023-11-24T06:28:41.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.118+0000 7f054ffff640 1 -- 172.21.15.64:0/3903701389 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0560040cc0 con 0x7f056815bf10 2023-11-24T06:28:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.119+0000 7f054ffff640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f055406ae40 0x7f055406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.119+0000 7f054ffff640 1 -- 172.21.15.64:0/3903701389 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f05600ac440 con 0x7f056815bf10 2023-11-24T06:28:41.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.120+0000 7f05677fe640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f055406ae40 0x7f055406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.121+0000 7f05677fe640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f055406ae40 0x7f055406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05580015f0 tx=0x7f0558007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.129+0000 7f054ffff640 1 -- 172.21.15.64:0/3903701389 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05600773f0 con 0x7f056815bf10 2023-11-24T06:28:41.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.411+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f056806c7a0 con 0x7f056815bf10 2023-11-24T06:28:41.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.412+0000 7f054ffff640 1 -- 172.21.15.64:0/3903701389 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0560009450 con 0x7f056815bf10 2023-11-24T06:28:41.416 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:41.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f055406ae40 msgr2=0x7f055406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f055406ae40 0x7f055406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05580015f0 tx=0x7f0558007a40 comp rx=0 tx=0).stop 2023-11-24T06:28:41.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 msgr2=0x7f056815a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f056815a7e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0560009e80 tx=0x7f0560004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:41.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 shutdown_connections 2023-11-24T06:28:41.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f055406ae40 0x7f055406d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f056815bf10 0x7f056815a7e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05680ccb50 0x7f05681f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 --2- 172.21.15.64:0/3903701389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05680adf20 0x7f05680b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 >> 172.21.15.64:0/3903701389 conn(0x7f056806a7d0 msgr2=0x7f056806db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 shutdown_connections 2023-11-24T06:28:41.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.415+0000 7f056cb73640 1 -- 172.21.15.64:0/3903701389 wait complete. 2023-11-24T06:28:41.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:41.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.677+0000 7fdbad333640 1 Processor -- start 2023-11-24T06:28:41.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.678+0000 7fdbad333640 1 -- start start 2023-11-24T06:28:41.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.679+0000 7fdbad333640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdba8143ae0 0x7fdba80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.679+0000 7fdbad333640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.680+0000 7fdbad333640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba80cfc30 con 0x7fdba815e1e0 2023-11-24T06:28:41.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.680+0000 7fdbad333640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba80b1ee0 con 0x7fdba8143ae0 2023-11-24T06:28:41.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.680+0000 7fdbad333640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b2780 con 0x7fdba80cf7d0 2023-11-24T06:28:41.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.680+0000 7fdba6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdba8143ae0 0x7fdba80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.680+0000 7fdba6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdba8143ae0 0x7fdba80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40384/0 (socket says 172.21.15.64:40384) 2023-11-24T06:28:41.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.680+0000 7fdba6ffd640 1 -- 172.21.15.64:0/1836351542 learned_addr learned my addr 172.21.15.64:0/1836351542 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:41.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.680+0000 7fdba67fc640 1 --2- 172.21.15.64:0/1836351542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.681+0000 7fdba6ffd640 1 -- 172.21.15.64:0/1836351542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 msgr2=0x7fdba815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:41.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.681+0000 7fdba6ffd640 1 --2- 172.21.15.64:0/1836351542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.681+0000 7fdba6ffd640 1 -- 172.21.15.64:0/1836351542 >> v1:172.21.15.162:6789/0 conn(0x7fdba815e1e0 legacy=0x7fdba815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.681+0000 7fdba6ffd640 1 -- 172.21.15.64:0/1836351542 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba815e990 con 0x7fdba8143ae0 2023-11-24T06:28:41.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.682+0000 7fdba6ffd640 1 --2- 172.21.15.64:0/1836351542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba8143ae0 0x7fdba80cf290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb9802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d31407be43cd9e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.682+0000 7fdba5ffb640 1 -- 172.21.15.64:0/1836351542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98030840 con 0x7fdba8143ae0 2023-11-24T06:28:41.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.682+0000 7fdba5ffb640 1 -- 172.21.15.64:0/1836351542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb980309a0 con 0x7fdba8143ae0 2023-11-24T06:28:41.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdba5ffb640 1 -- 172.21.15.64:0/1836351542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98030d30 con 0x7fdba8143ae0 2023-11-24T06:28:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdbad333640 1 -- 172.21.15.64:0/1836351542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba8143ae0 msgr2=0x7fdba80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdbad333640 1 --2- 172.21.15.64:0/1836351542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba8143ae0 0x7fdba80cf290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb9802b980 comp rx=0 tx=0).stop 2023-11-24T06:28:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdbad333640 1 -- 172.21.15.64:0/1836351542 shutdown_connections 2023-11-24T06:28:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdbad333640 1 --2- 172.21.15.64:0/1836351542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdbad333640 1 --2- 172.21.15.64:0/1836351542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba8143ae0 0x7fdba80cf290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdbad333640 1 -- 172.21.15.64:0/1836351542 >> 172.21.15.64:0/1836351542 conn(0x7fdba806a800 msgr2=0x7fdba806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.683+0000 7fdbad333640 1 -- 172.21.15.64:0/1836351542 shutdown_connections 2023-11-24T06:28:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.684+0000 7fdbad333640 1 -- 172.21.15.64:0/1836351542 wait complete. 2023-11-24T06:28:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.685+0000 7fdbad333640 1 Processor -- start 2023-11-24T06:28:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.685+0000 7fdbad333640 1 -- start start 2023-11-24T06:28:41.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdbad333640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba8142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdbad333640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba8143ae0 0x7fdba8141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdbad333640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba815e1e0 0x7fdba815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdbad333640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba81e3d00 con 0x7fdba815e1e0 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdbad333640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba815c480 con 0x7fdba8143ae0 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdbad333640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba815c700 con 0x7fdba80cf7d0 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdba6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba8142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.686+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba815e1e0 0x7fdba815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba8142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44268/0 (socket says 172.21.15.64:44268) 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba6ffd640 1 -- 172.21.15.64:0/2056019049 learned_addr learned my addr 172.21.15.64:0/2056019049 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba67fc640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba8143ae0 0x7fdba8141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba77fe640 1 -- 172.21.15.64:0/2056019049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 msgr2=0x7fdba8142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba77fe640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba8142350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba77fe640 1 -- 172.21.15.64:0/2056019049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba8143ae0 msgr2=0x7fdba8141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba77fe640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba8143ae0 0x7fdba8141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba77fe640 1 -- 172.21.15.64:0/2056019049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980094d0 con 0x7fdba815e1e0 2023-11-24T06:28:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdba77fe640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba815e1e0 0x7fdba815bf40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c700 tx=0x7fdba000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.687+0000 7fdb8f7fe640 1 -- 172.21.15.64:0/2056019049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003b70 con 0x7fdba815e1e0 2023-11-24T06:28:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.688+0000 7fdb8f7fe640 1 -- 172.21.15.64:0/2056019049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0003cd0 con 0x7fdba815e1e0 2023-11-24T06:28:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.688+0000 7fdb8f7fe640 1 -- 172.21.15.64:0/2056019049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0005310 con 0x7fdba815e1e0 2023-11-24T06:28:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.688+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba815c9e0 con 0x7fdba815e1e0 2023-11-24T06:28:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.688+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba81e3f10 con 0x7fdba815e1e0 2023-11-24T06:28:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.690+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba80b2bf0 con 0x7fdba815e1e0 2023-11-24T06:28:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.691+0000 7fdb8f7fe640 1 -- 172.21.15.64:0/2056019049 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdba0005470 con 0x7fdba815e1e0 2023-11-24T06:28:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.692+0000 7fdb8f7fe640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdb9406af10 0x7fdb9406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.693+0000 7fdb8f7fe640 1 -- 172.21.15.64:0/2056019049 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fdba00891f0 con 0x7fdba815e1e0 2023-11-24T06:28:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.693+0000 7fdba6ffd640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdb9406af10 0x7fdb9406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.694+0000 7fdba6ffd640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdb9406af10 0x7fdb9406d3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb98008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.697+0000 7fdb8f7fe640 1 -- 172.21.15.64:0/2056019049 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdba80b2bf0 con 0x7fdba815e1e0 2023-11-24T06:28:41.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.934+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdba80b2bf0 con 0x7fdba815e1e0 2023-11-24T06:28:41.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.935+0000 7fdb8f7fe640 1 -- 172.21.15.64:0/2056019049 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdba80b2bf0 con 0x7fdba815e1e0 2023-11-24T06:28:41.939 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdb9406af10 msgr2=0x7fdb9406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdb9406af10 0x7fdb9406d3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb98008030 comp rx=0 tx=0).stop 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba815e1e0 msgr2=0x7fdba815bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba815e1e0 0x7fdba815bf40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c700 tx=0x7fdba000cbc0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 shutdown_connections 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdb9406af10 0x7fdb9406d3d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdba815e1e0 0x7fdba815bf40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba8143ae0 0x7fdba8141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 --2- 172.21.15.64:0/2056019049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba80cf7d0 0x7fdba8142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 >> 172.21.15.64:0/2056019049 conn(0x7fdba806a800 msgr2=0x7fdba80ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 shutdown_connections 2023-11-24T06:28:41.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:41.938+0000 7fdbad333640 1 -- 172.21.15.64:0/2056019049 wait complete. 2023-11-24T06:28:41.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:42.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.195+0000 7fecc9c39640 1 Processor -- start 2023-11-24T06:28:42.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.196+0000 7fecc9c39640 1 -- start start 2023-11-24T06:28:42.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.197+0000 7fecc9c39640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fecc413fcb0 0x7fecc40cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.197+0000 7fecc9c39640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc40ce440 con 0x7fecc40cdc20 2023-11-24T06:28:42.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.198+0000 7fecc9c39640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc40ce720 con 0x7fecc413cf10 2023-11-24T06:28:42.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.198+0000 7fecc9c39640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc40cea00 con 0x7fecc413fcb0 2023-11-24T06:28:42.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.198+0000 7fecbbfff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fecc413fcb0 0x7fecc40cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.198+0000 7fecc8c37640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fecc413cf10 0x7fecc413f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:49498/0 (socket says 172.21.15.64:49498) 2023-11-24T06:28:42.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.198+0000 7fecc8c37640 1 -- 172.21.15.64:0/894669321 learned_addr learned my addr 172.21.15.64:0/894669321 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:42.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.199+0000 7fecbb7fe640 1 -- 172.21.15.64:0/894669321 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657587916 0 0) 0x7fecc40ce720 con 0x7fecc413cf10 2023-11-24T06:28:42.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.199+0000 7fecbbfff640 1 -- 172.21.15.64:0/894669321 >> v1:172.21.15.162:6789/0 conn(0x7fecc413cf10 legacy=0x7fecc413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.199+0000 7fecbbfff640 1 -- 172.21.15.64:0/894669321 >> v1:172.21.15.64:6789/0 conn(0x7fecc40cdc20 legacy=0x7fecc40b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.199+0000 7fecbbfff640 1 -- 172.21.15.64:0/894669321 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc40b1d90 con 0x7fecc413fcb0 2023-11-24T06:28:42.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.201+0000 7fecbbfff640 1 --2- 172.21.15.64:0/894669321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc413fcb0 0x7fecc40cd6e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fecb40097a0 tx=0x7fecb402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75f4f956f4695963 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.201+0000 7fecbb7fe640 1 -- 172.21.15.64:0/894669321 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4030710 con 0x7fecc413fcb0 2023-11-24T06:28:42.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.201+0000 7fecbb7fe640 1 -- 172.21.15.64:0/894669321 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb402e050 con 0x7fecc413fcb0 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.201+0000 7fecbb7fe640 1 -- 172.21.15.64:0/894669321 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4030a90 con 0x7fecc413fcb0 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.202+0000 7fecc9c39640 1 -- 172.21.15.64:0/894669321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc413fcb0 msgr2=0x7fecc40cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.202+0000 7fecc9c39640 1 --2- 172.21.15.64:0/894669321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc413fcb0 0x7fecc40cd6e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fecb40097a0 tx=0x7fecb402b790 comp rx=0 tx=0).stop 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.202+0000 7fecc9c39640 1 -- 172.21.15.64:0/894669321 shutdown_connections 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.202+0000 7fecc9c39640 1 --2- 172.21.15.64:0/894669321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc413fcb0 0x7fecc40cd6e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.202+0000 7fecc9c39640 1 -- 172.21.15.64:0/894669321 >> 172.21.15.64:0/894669321 conn(0x7fecc406a7d0 msgr2=0x7fecc406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.202+0000 7fecc9c39640 1 -- 172.21.15.64:0/894669321 shutdown_connections 2023-11-24T06:28:42.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.202+0000 7fecc9c39640 1 -- 172.21.15.64:0/894669321 wait complete. 2023-11-24T06:28:42.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.204+0000 7fecc9c39640 1 Processor -- start 2023-11-24T06:28:42.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.204+0000 7fecc9c39640 1 -- start start 2023-11-24T06:28:42.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.205+0000 7fecc9c39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc40cdc20 0x7fecc41384a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.205+0000 7fecc9c39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc413cf10 0x7fecc413e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.205+0000 7fecc9c39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecc413ecb0 0x7fecc41f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.205+0000 7fecc9c39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc413e2e0 con 0x7fecc413cf10 2023-11-24T06:28:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.205+0000 7fecc9c39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc413d6d0 con 0x7fecc40cdc20 2023-11-24T06:28:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.205+0000 7fecc9c39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc413d9b0 con 0x7fecc413ecb0 2023-11-24T06:28:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.206+0000 7fecc8c37640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc40cdc20 0x7fecc41384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.206+0000 7fecc9438640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecc413ecb0 0x7fecc41f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.206+0000 7fecc8c37640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc40cdc20 0x7fecc41384a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:54938/0 (socket says 172.21.15.64:54938) 2023-11-24T06:28:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.206+0000 7fecc8c37640 1 -- 172.21.15.64:0/2187393933 learned_addr learned my addr 172.21.15.64:0/2187393933 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.206+0000 7fecbbfff640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc413cf10 0x7fecc413e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.207+0000 7fecc8c37640 1 -- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecc413ecb0 msgr2=0x7fecc41f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.207+0000 7fecc8c37640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecc413ecb0 0x7fecc41f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.207+0000 7fecc8c37640 1 -- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc413cf10 msgr2=0x7fecc413e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.207+0000 7fecc8c37640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc413cf10 0x7fecc413e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.207+0000 7fecc8c37640 1 -- 172.21.15.64:0/2187393933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4009400 con 0x7fecc40cdc20 2023-11-24T06:28:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.207+0000 7fecc9438640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecc413ecb0 0x7fecc41f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.207+0000 7fecc8c37640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc40cdc20 0x7fecc41384a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c430 tx=0x7fecc000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.208+0000 7fecb97fa640 1 -- 172.21.15.64:0/2187393933 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00074e0 con 0x7fecc40cdc20 2023-11-24T06:28:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.208+0000 7fecb97fa640 1 -- 172.21.15.64:0/2187393933 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc0007660 con 0x7fecc40cdc20 2023-11-24T06:28:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.208+0000 7fecb97fa640 1 -- 172.21.15.64:0/2187393933 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0004a20 con 0x7fecc40cdc20 2023-11-24T06:28:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.208+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc413dc90 con 0x7fecc40cdc20 2023-11-24T06:28:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.208+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc413d2f0 con 0x7fecc40cdc20 2023-11-24T06:28:42.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.209+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc4067450 con 0x7fecc40cdc20 2023-11-24T06:28:42.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.211+0000 7fecb97fa640 1 -- 172.21.15.64:0/2187393933 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecc0004b80 con 0x7fecc40cdc20 2023-11-24T06:28:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.213+0000 7fecb97fa640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fecb006ae40 0x7fecb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.213+0000 7fecb97fa640 1 -- 172.21.15.64:0/2187393933 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fecc0089480 con 0x7fecc40cdc20 2023-11-24T06:28:42.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.213+0000 7fecbbfff640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fecb006ae40 0x7fecb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.214+0000 7fecbbfff640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fecb006ae40 0x7fecb006d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fecb4034000 tx=0x7fecb402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.227+0000 7fecb97fa640 1 -- 172.21.15.64:0/2187393933 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecc0054450 con 0x7fecc40cdc20 2023-11-24T06:28:42.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.493+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fecc40b3f40 con 0x7fecc40cdc20 2023-11-24T06:28:42.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.493+0000 7fecb97fa640 1 -- 172.21.15.64:0/2187393933 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecc0057a70 con 0x7fecc40cdc20 2023-11-24T06:28:42.497 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:42.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.496+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fecb006ae40 msgr2=0x7fecb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.496+0000 7fecc9c39640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fecb006ae40 0x7fecb006d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fecb4034000 tx=0x7fecb402f000 comp rx=0 tx=0).stop 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc40cdc20 msgr2=0x7fecc41384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc40cdc20 0x7fecc41384a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c430 tx=0x7fecc000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 shutdown_connections 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fecb006ae40 0x7fecb006d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecc413ecb0 0x7fecc41f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc413cf10 0x7fecc413e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 --2- 172.21.15.64:0/2187393933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc40cdc20 0x7fecc41384a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 >> 172.21.15.64:0/2187393933 conn(0x7fecc406a7d0 msgr2=0x7fecc4158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 shutdown_connections 2023-11-24T06:28:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.497+0000 7fecc9c39640 1 -- 172.21.15.64:0/2187393933 wait complete. 2023-11-24T06:28:42.519 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:42.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.758+0000 7ffa2e467640 1 Processor -- start 2023-11-24T06:28:42.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.759+0000 7ffa2e467640 1 -- start start 2023-11-24T06:28:42.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.760+0000 7ffa2e467640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffa2815b100 0x7ffa280cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2e467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2e467640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa280b1e90 con 0x7ffa280ce460 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2e467640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa280cbf20 con 0x7ffa2815b100 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2e467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2815b4e0 con 0x7ffa280d0bb0 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2cc64640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa280ce460 0x7ffa280b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41686/0 (socket says 172.21.15.64:41686) 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2cc64640 1 -- 172.21.15.64:0/1689955041 learned_addr learned my addr 172.21.15.64:0/1689955041 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2d465640 1 --2- 172.21.15.64:0/1689955041 >> v2:172.21.15.64:3300/0 conn(0x7ffa2815b100 0x7ffa280cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.761+0000 7ffa2dc66640 1 --2- 172.21.15.64:0/1689955041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.762+0000 7ffa1ffff640 1 -- 172.21.15.64:0/1689955041 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361618570 0 0) 0x7ffa280b1e90 con 0x7ffa280ce460 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.762+0000 7ffa1ffff640 1 -- 172.21.15.64:0/1689955041 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa14003520 con 0x7ffa280ce460 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.762+0000 7ffa2d465640 1 -- 172.21.15.64:0/1689955041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 msgr2=0x7ffa280d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.762+0000 7ffa2d465640 1 --2- 172.21.15.64:0/1689955041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.762+0000 7ffa2d465640 1 -- 172.21.15.64:0/1689955041 >> v1:172.21.15.64:6789/0 conn(0x7ffa280ce460 legacy=0x7ffa280b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.762+0000 7ffa2d465640 1 -- 172.21.15.64:0/1689955041 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa280d1580 con 0x7ffa2815b100 2023-11-24T06:28:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.762+0000 7ffa2dc66640 1 --2- 172.21.15.64:0/1689955041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.763+0000 7ffa2d465640 1 --2- 172.21.15.64:0/1689955041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa2815b100 0x7ffa280cb9e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffa240097d0 tx=0x7ffa2402b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb889b5cfd25ef6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.764+0000 7ffa1ffff640 1 -- 172.21.15.64:0/1689955041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24031770 con 0x7ffa2815b100 2023-11-24T06:28:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.764+0000 7ffa1ffff640 1 -- 172.21.15.64:0/1689955041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa240318d0 con 0x7ffa2815b100 2023-11-24T06:28:42.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.764+0000 7ffa1ffff640 1 -- 172.21.15.64:0/1689955041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24031c40 con 0x7ffa2815b100 2023-11-24T06:28:42.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.764+0000 7ffa2e467640 1 -- 172.21.15.64:0/1689955041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa2815b100 msgr2=0x7ffa280cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.764+0000 7ffa2e467640 1 --2- 172.21.15.64:0/1689955041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa2815b100 0x7ffa280cb9e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffa240097d0 tx=0x7ffa2402b810 comp rx=0 tx=0).stop 2023-11-24T06:28:42.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.765+0000 7ffa2e467640 1 -- 172.21.15.64:0/1689955041 shutdown_connections 2023-11-24T06:28:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.765+0000 7ffa2e467640 1 --2- 172.21.15.64:0/1689955041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.765+0000 7ffa2e467640 1 --2- 172.21.15.64:0/1689955041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa2815b100 0x7ffa280cb9e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.765+0000 7ffa2e467640 1 -- 172.21.15.64:0/1689955041 >> 172.21.15.64:0/1689955041 conn(0x7ffa28099a60 msgr2=0x7ffa28099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.765+0000 7ffa2e467640 1 -- 172.21.15.64:0/1689955041 shutdown_connections 2023-11-24T06:28:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.765+0000 7ffa2e467640 1 -- 172.21.15.64:0/1689955041 wait complete. 2023-11-24T06:28:42.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.767+0000 7ffa2e467640 1 Processor -- start 2023-11-24T06:28:42.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.767+0000 7ffa2e467640 1 -- start start 2023-11-24T06:28:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.768+0000 7ffa2e467640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa280ce460 0x7ffa2814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.768+0000 7ffa2e467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2d465640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa280ce460 0x7ffa2814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2e467640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2815b100 0x7ffa280cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2e467640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280c4990 con 0x7ffa280ce460 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2e467640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280c4c30 con 0x7ffa2815b100 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2e467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280c4f10 con 0x7ffa280d0bb0 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2d465640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa280ce460 0x7ffa2814b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40428/0 (socket says 172.21.15.64:40428) 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2d465640 1 -- 172.21.15.64:0/2063062493 learned_addr learned my addr 172.21.15.64:0/2063062493 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2cc64640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.769+0000 7ffa2dc66640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2815b100 0x7ffa280cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.770+0000 7ffa2d465640 1 -- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 msgr2=0x7ffa280cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.770+0000 7ffa2d465640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.770+0000 7ffa2d465640 1 -- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2815b100 msgr2=0x7ffa280cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.770+0000 7ffa2d465640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2815b100 0x7ffa280cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.770+0000 7ffa2d465640 1 -- 172.21.15.64:0/2063062493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa20009a70 con 0x7ffa280ce460 2023-11-24T06:28:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.770+0000 7ffa2cc64640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.771+0000 7ffa2d465640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa280ce460 0x7ffa2814b760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffa24000c00 tx=0x7ffa240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.771+0000 7ffa1dffb640 1 -- 172.21.15.64:0/2063062493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24032610 con 0x7ffa280ce460 2023-11-24T06:28:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.771+0000 7ffa1dffb640 1 -- 172.21.15.64:0/2063062493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24032770 con 0x7ffa280ce460 2023-11-24T06:28:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.771+0000 7ffa1dffb640 1 -- 172.21.15.64:0/2063062493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24032b20 con 0x7ffa280ce460 2023-11-24T06:28:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.771+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa24009400 con 0x7ffa280ce460 2023-11-24T06:28:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.771+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa280c54d0 con 0x7ffa280ce460 2023-11-24T06:28:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.773+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa280d1b40 con 0x7ffa280ce460 2023-11-24T06:28:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.775+0000 7ffa1dffb640 1 -- 172.21.15.64:0/2063062493 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa24032c80 con 0x7ffa280ce460 2023-11-24T06:28:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.776+0000 7ffa1dffb640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa0c06ae40 0x7ffa0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.776+0000 7ffa1dffb640 1 -- 172.21.15.64:0/2063062493 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7ffa240ab350 con 0x7ffa280ce460 2023-11-24T06:28:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.776+0000 7ffa2cc64640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa0c06ae40 0x7ffa0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.778+0000 7ffa2cc64640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa0c06ae40 0x7ffa0c06d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa281f6560 tx=0x7ffa18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:42.783+0000 7ffa1dffb640 1 -- 172.21.15.64:0/2063062493 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa24076430 con 0x7ffa280ce460 2023-11-24T06:28:43.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.069+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa28067b60 con 0x7ffa280ce460 2023-11-24T06:28:43.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.069+0000 7ffa1dffb640 1 -- 172.21.15.64:0/2063062493 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffa24079a50 con 0x7ffa280ce460 2023-11-24T06:28:43.073 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:43.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.072+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa0c06ae40 msgr2=0x7ffa0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.072+0000 7ffa2e467640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa0c06ae40 0x7ffa0c06d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa281f6560 tx=0x7ffa18009040 comp rx=0 tx=0).stop 2023-11-24T06:28:43.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.072+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa280ce460 msgr2=0x7ffa2814b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.072+0000 7ffa2e467640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa280ce460 0x7ffa2814b760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffa24000c00 tx=0x7ffa240046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 shutdown_connections 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ffa0c06ae40 0x7ffa0c06d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa2815b100 0x7ffa280cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa280d0bb0 0x7ffa280cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 --2- 172.21.15.64:0/2063062493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa280ce460 0x7ffa2814b760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 >> 172.21.15.64:0/2063062493 conn(0x7ffa28099a60 msgr2=0x7ffa2806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 shutdown_connections 2023-11-24T06:28:43.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.073+0000 7ffa2e467640 1 -- 172.21.15.64:0/2063062493 wait complete. 2023-11-24T06:28:43.096 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:28:43.096 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:43.136 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:43.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.368+0000 7f6b90be4640 1 Processor -- start 2023-11-24T06:28:43.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.368+0000 7f6b90be4640 1 -- start start 2023-11-24T06:28:43.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.369+0000 7f6b90be4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b8c0b3f30 0x7f6b8c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.369+0000 7f6b90be4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c0d0a10 con 0x7f6b8c160920 2023-11-24T06:28:43.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.369+0000 7f6b90be4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c0cf080 con 0x7f6b8c0d0100 2023-11-24T06:28:43.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.369+0000 7f6b90be4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b8c0c6be0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.370+0000 7f6b8bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b8c0b3f30 0x7f6b8c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.370+0000 7f6b8bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b8c0b3f30 0x7f6b8c0d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33688/0 (socket says 172.21.15.64:33688) 2023-11-24T06:28:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.370+0000 7f6b8bfff640 1 -- 172.21.15.64:0/2494401182 learned_addr learned my addr 172.21.15.64:0/2494401182 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.370+0000 7f6b8bfff640 1 -- 172.21.15.64:0/2494401182 >> v1:172.21.15.162:6789/0 conn(0x7f6b8c0d0100 legacy=0x7f6b8c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.370+0000 7f6b8bfff640 1 -- 172.21.15.64:0/2494401182 >> v1:172.21.15.64:6789/0 conn(0x7f6b8c160920 legacy=0x7f6b8c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.370+0000 7f6b8bfff640 1 -- 172.21.15.64:0/2494401182 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0ce7e0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.371+0000 7f6b8bfff640 1 --2- 172.21.15.64:0/2494401182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c0d1650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009e50 tx=0x7f6b8402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ea62ae1e41f183 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.372+0000 7f6b8a7fc640 1 -- 172.21.15.64:0/2494401182 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8402cbb0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.372+0000 7f6b8a7fc640 1 -- 172.21.15.64:0/2494401182 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8402cd10 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.372+0000 7f6b8a7fc640 1 -- 172.21.15.64:0/2494401182 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84034610 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.372+0000 7f6b90be4640 1 -- 172.21.15.64:0/2494401182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 msgr2=0x7f6b8c0d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.372+0000 7f6b90be4640 1 --2- 172.21.15.64:0/2494401182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c0d1650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009e50 tx=0x7f6b8402c010 comp rx=0 tx=0).stop 2023-11-24T06:28:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.373+0000 7f6b90be4640 1 -- 172.21.15.64:0/2494401182 shutdown_connections 2023-11-24T06:28:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.373+0000 7f6b90be4640 1 --2- 172.21.15.64:0/2494401182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c0d1650 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.373+0000 7f6b90be4640 1 -- 172.21.15.64:0/2494401182 >> 172.21.15.64:0/2494401182 conn(0x7f6b8c0d3090 msgr2=0x7f6b8c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.373+0000 7f6b90be4640 1 -- 172.21.15.64:0/2494401182 shutdown_connections 2023-11-24T06:28:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.373+0000 7f6b90be4640 1 -- 172.21.15.64:0/2494401182 wait complete. 2023-11-24T06:28:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.375+0000 7f6b90be4640 1 Processor -- start 2023-11-24T06:28:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.375+0000 7f6b90be4640 1 -- start start 2023-11-24T06:28:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b90be4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b90be4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b8c0d0100 0x7f6b8c1f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b90be4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8c160920 0x7f6b8c159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b90be4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0b8ef0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b90be4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0b8c10 con 0x7f6b8c160920 2023-11-24T06:28:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b90be4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0cb9a0 con 0x7f6b8c0d0100 2023-11-24T06:28:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33692/0 (socket says 172.21.15.64:33692) 2023-11-24T06:28:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b8b7fe640 1 -- 172.21.15.64:0/542730323 learned_addr learned my addr 172.21.15.64:0/542730323 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b8affd640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b8c0d0100 0x7f6b8c1f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.377+0000 7f6b8bfff640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8c160920 0x7f6b8c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b8b7fe640 1 -- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b8c0d0100 msgr2=0x7f6b8c1f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b8b7fe640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b8c0d0100 0x7f6b8c1f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b8b7fe640 1 -- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8c160920 msgr2=0x7f6b8c159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b8b7fe640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8c160920 0x7f6b8c159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b8b7fe640 1 -- 172.21.15.64:0/542730323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84009b30 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b8b7fe640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c158bd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c4d0 tx=0x7f6b7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b73fff640 1 -- 172.21.15.64:0/542730323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c015e30 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b73fff640 1 -- 172.21.15.64:0/542730323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c0050d0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c0cbc80 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.378+0000 7f6b73fff640 1 -- 172.21.15.64:0/542730323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c005480 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.379+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c159110 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.380+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c0cc1a0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.381+0000 7f6b73fff640 1 -- 172.21.15.64:0/542730323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b7c0035f0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.382+0000 7f6b73fff640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b7806ae40 0x7f6b7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.382+0000 7f6b73fff640 1 -- 172.21.15.64:0/542730323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f6b7c091280 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.383+0000 7f6b8affd640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b7806ae40 0x7f6b7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.384+0000 7f6b8affd640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b7806ae40 0x7f6b7806d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0c67d0 tx=0x7f6b80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.386+0000 7f6b73fff640 1 -- 172.21.15.64:0/542730323 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b8c0cc1a0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.664+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b8c0cc1a0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.666+0000 7f6b73fff640 1 -- 172.21.15.64:0/542730323 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f6b8c0cc1a0 con 0x7f6b8c0b3f30 2023-11-24T06:28:43.669 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:43.670 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b7806ae40 msgr2=0x7f6b7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b7806ae40 0x7f6b7806d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0c67d0 tx=0x7f6b80009040 comp rx=0 tx=0).stop 2023-11-24T06:28:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 msgr2=0x7f6b8c158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c158bd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c4d0 tx=0x7f6b7c00c990 comp rx=0 tx=0).stop 2023-11-24T06:28:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 shutdown_connections 2023-11-24T06:28:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b7806ae40 0x7f6b7806d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8c160920 0x7f6b8c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b8c0d0100 0x7f6b8c1f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 --2- 172.21.15.64:0/542730323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c158bd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 >> 172.21.15.64:0/542730323 conn(0x7f6b8c0d3090 msgr2=0x7f6b8c0b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 shutdown_connections 2023-11-24T06:28:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.668+0000 7f6b90be4640 1 -- 172.21.15.64:0/542730323 wait complete. 2023-11-24T06:28:43.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:43.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:43.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.881+0000 7f935986e640 1 Processor -- start 2023-11-24T06:28:43.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.882+0000 7f935986e640 1 -- start start 2023-11-24T06:28:43.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.883+0000 7f935986e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f93540cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.883+0000 7f935986e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93540d0320 con 0x7f935415d4b0 2023-11-24T06:28:43.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.883+0000 7f935986e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93540d1c60 con 0x7f93540b31d0 2023-11-24T06:28:43.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.883+0000 7f935986e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93540b2730 con 0x7f93540cf9a0 2023-11-24T06:28:43.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.883+0000 7f935886c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f935415d4b0 0x7f93540d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34812/0 (socket says 172.21.15.64:34812) 2023-11-24T06:28:43.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.883+0000 7f935886c640 1 -- 172.21.15.64:0/1691975250 learned_addr learned my addr 172.21.15.64:0/1691975250 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:43.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.884+0000 7f9353fff640 1 --2- 172.21.15.64:0/1691975250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f93540cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.884+0000 7f93537fe640 1 -- 172.21.15.64:0/1691975250 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007272852 0 0) 0x7f93540d0320 con 0x7f935415d4b0 2023-11-24T06:28:43.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.884+0000 7f93537fe640 1 -- 172.21.15.64:0/1691975250 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9344003520 con 0x7f935415d4b0 2023-11-24T06:28:43.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.884+0000 7f9353fff640 1 -- 172.21.15.64:0/1691975250 >> v1:172.21.15.162:6789/0 conn(0x7f93540b31d0 legacy=0x7f93540b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.884+0000 7f93537fe640 1 -- 172.21.15.64:0/1691975250 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443931141 0 0) 0x7f93540d1c60 con 0x7f93540b31d0 2023-11-24T06:28:43.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.884+0000 7f9353fff640 1 -- 172.21.15.64:0/1691975250 >> v1:172.21.15.64:6789/0 conn(0x7f935415d4b0 legacy=0x7f93540d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.884+0000 7f9353fff640 1 -- 172.21.15.64:0/1691975250 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540d1ec0 con 0x7f93540cf9a0 2023-11-24T06:28:43.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.886+0000 7f9353fff640 1 --2- 172.21.15.64:0/1691975250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f93540cfde0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9340009800 tx=0x7f934002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720c569c47e4dbf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.886+0000 7f93537fe640 1 -- 172.21.15.64:0/1691975250 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031890 con 0x7f93540cf9a0 2023-11-24T06:28:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.886+0000 7f93537fe640 1 -- 172.21.15.64:0/1691975250 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93400319f0 con 0x7f93540cf9a0 2023-11-24T06:28:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.886+0000 7f93537fe640 1 -- 172.21.15.64:0/1691975250 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031d80 con 0x7f93540cf9a0 2023-11-24T06:28:43.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.887+0000 7f935986e640 1 -- 172.21.15.64:0/1691975250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 msgr2=0x7f93540cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.887+0000 7f935986e640 1 --2- 172.21.15.64:0/1691975250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f93540cfde0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9340009800 tx=0x7f934002b930 comp rx=0 tx=0).stop 2023-11-24T06:28:43.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.887+0000 7f935986e640 1 -- 172.21.15.64:0/1691975250 shutdown_connections 2023-11-24T06:28:43.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.887+0000 7f935986e640 1 --2- 172.21.15.64:0/1691975250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f93540cfde0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.887+0000 7f935986e640 1 -- 172.21.15.64:0/1691975250 >> 172.21.15.64:0/1691975250 conn(0x7f93540d3000 msgr2=0x7f93540d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:43.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.887+0000 7f935986e640 1 -- 172.21.15.64:0/1691975250 shutdown_connections 2023-11-24T06:28:43.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.887+0000 7f935986e640 1 -- 172.21.15.64:0/1691975250 wait complete. 2023-11-24T06:28:43.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.889+0000 7f935986e640 1 Processor -- start 2023-11-24T06:28:43.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.890+0000 7f935986e640 1 -- start start 2023-11-24T06:28:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.890+0000 7f935986e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93540b31d0 0x7f9354159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.890+0000 7f935986e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f935409d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.890+0000 7f935986e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935415d4b0 0x7f935409f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.890+0000 7f935986e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540a0230 con 0x7f93540b31d0 2023-11-24T06:28:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.890+0000 7f935986e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540a0510 con 0x7f935415d4b0 2023-11-24T06:28:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.890+0000 7f935986e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93540a07f0 con 0x7f93540cf9a0 2023-11-24T06:28:43.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f935886c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93540b31d0 0x7f9354159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f9353fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f935409d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f9353fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f935409d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44148/0 (socket says 172.21.15.64:44148) 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f9353fff640 1 -- 172.21.15.64:0/1970013958 learned_addr learned my addr 172.21.15.64:0/1970013958 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f935886c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93540b31d0 0x7f9354159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33694/0 (socket says 172.21.15.64:33694) 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f935906d640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935415d4b0 0x7f935409f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f935886c640 1 -- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 msgr2=0x7f935409d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.891+0000 7f935886c640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f935409d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f935886c640 1 -- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935415d4b0 msgr2=0x7f935409f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f935886c640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935415d4b0 0x7f935409f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f935886c640 1 -- 172.21.15.64:0/1970013958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009400 con 0x7f93540b31d0 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f9353fff640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f935409d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f935886c640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93540b31d0 0x7f9354159a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f934800c390 tx=0x7f9348001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f93517fa640 1 -- 172.21.15.64:0/1970013958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348005ce0 con 0x7f93540b31d0 2023-11-24T06:28:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93540a0ad0 con 0x7f93540b31d0 2023-11-24T06:28:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f93517fa640 1 -- 172.21.15.64:0/1970013958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348005e40 con 0x7f93540b31d0 2023-11-24T06:28:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f93517fa640 1 -- 172.21.15.64:0/1970013958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480072f0 con 0x7f93540b31d0 2023-11-24T06:28:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.892+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935409dba0 con 0x7f93540b31d0 2023-11-24T06:28:43.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.894+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935409ddf0 con 0x7f93540b31d0 2023-11-24T06:28:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.896+0000 7f93517fa640 1 -- 172.21.15.64:0/1970013958 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9348007510 con 0x7f93540b31d0 2023-11-24T06:28:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.896+0000 7f93517fa640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f933806ae40 0x7f933806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.897+0000 7f93517fa640 1 -- 172.21.15.64:0/1970013958 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f9348088a60 con 0x7f93540b31d0 2023-11-24T06:28:43.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.897+0000 7f9353fff640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f933806ae40 0x7f933806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.898+0000 7f9353fff640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f933806ae40 0x7f933806d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f935415cc90 tx=0x7f934003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:43.903+0000 7f93517fa640 1 -- 172.21.15.64:0/1970013958 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9348053b70 con 0x7f93540b31d0 2023-11-24T06:28:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.165+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93540cf0c0 con 0x7f93540b31d0 2023-11-24T06:28:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.166+0000 7f93517fa640 1 -- 172.21.15.64:0/1970013958 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93480170a0 con 0x7f93540b31d0 2023-11-24T06:28:44.170 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.169+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f933806ae40 msgr2=0x7f933806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.169+0000 7f935986e640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f933806ae40 0x7f933806d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f935415cc90 tx=0x7f934003d040 comp rx=0 tx=0).stop 2023-11-24T06:28:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.169+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93540b31d0 msgr2=0x7f9354159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.169+0000 7f935986e640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93540b31d0 0x7f9354159a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f934800c390 tx=0x7f9348001e70 comp rx=0 tx=0).stop 2023-11-24T06:28:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 shutdown_connections 2023-11-24T06:28:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f933806ae40 0x7f933806d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935415d4b0 0x7f935409f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93540cf9a0 0x7f935409d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 --2- 172.21.15.64:0/1970013958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93540b31d0 0x7f9354159a50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 >> 172.21.15.64:0/1970013958 conn(0x7f93540d3000 msgr2=0x7f935406ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 shutdown_connections 2023-11-24T06:28:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.170+0000 7f935986e640 1 -- 172.21.15.64:0/1970013958 wait complete. 2023-11-24T06:28:44.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:44.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.403+0000 7fa1325c9640 1 Processor -- start 2023-11-24T06:28:44.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.403+0000 7fa1325c9640 1 -- start start 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.405+0000 7fa1325c9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa12c0b3c50 0x7fa12c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.405+0000 7fa1325c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c0cd7e0 0x7fa12c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.405+0000 7fa1325c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12c0cb660 con 0x7fa12c0b2480 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.405+0000 7fa1325c9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c0ce1b0 con 0x7fa12c0b3c50 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.405+0000 7fa1325c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0cb450 con 0x7fa12c0cd7e0 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.405+0000 7fa1315c7640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa12c0b2480 0x7fa12c0d01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34816/0 (socket says 172.21.15.64:34816) 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.405+0000 7fa1315c7640 1 -- 172.21.15.64:0/3714174521 learned_addr learned my addr 172.21.15.64:0/3714174521 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:44.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3714174521 >> v2:172.21.15.162:3300/0 conn(0x7fa12c0b3c50 0x7fa12c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa131dc8640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c0cd7e0 0x7fa12c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747495302 0 0) 0x7fa12c0cb660 con 0x7fa12c0b2480 2023-11-24T06:28:44.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa118003520 con 0x7fa12c0b2480 2023-11-24T06:28:44.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927532421 0 0) 0x7fa118003520 con 0x7fa12c0b2480 2023-11-24T06:28:44.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c0cd7e0 msgr2=0x7fa12c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:44.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c0cd7e0 0x7fa12c0cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 >> v2:172.21.15.162:3300/0 conn(0x7fa12c0b3c50 msgr2=0x7fa12c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 --2- 172.21.15.64:0/3714174521 >> v2:172.21.15.162:3300/0 conn(0x7fa12c0b3c50 0x7fa12c0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0cb940 con 0x7fa12c0b2480 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fa128001670 con 0x7fa12c0b2480 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 >> v1:172.21.15.64:6789/0 conn(0x7fa12c0b2480 legacy=0x7fa12c0d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa118003970 0x7fa118003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180047c0 0x7fa118004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa118005180 0x7fa11800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0ce1b0 con 0x7fa1180047c0 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.406+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0cb450 con 0x7fa118005180 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11800bf40 con 0x7fa118003970 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180047c0 0x7fa118004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa1315c7640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa118003970 0x7fa118003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa131dc8640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa118005180 0x7fa11800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa130dc6640 1 -- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa118003970 msgr2=0x7fa118003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:44.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa118003970 0x7fa118003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa130dc6640 1 -- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa118005180 msgr2=0x7fa11800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:44.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa118005180 0x7fa11800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.407+0000 7fa130dc6640 1 -- 172.21.15.64:0/3714174521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118003520 con 0x7fa1180047c0 2023-11-24T06:28:44.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.408+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180047c0 0x7fa118004c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009ac0 tx=0x7fa1180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0374e343d03f8f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.409+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c02c6f0 con 0x7fa1180047c0 2023-11-24T06:28:44.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.409+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c02c850 con 0x7fa1180047c0 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.409+0000 7fa123fff640 1 -- 172.21.15.64:0/3714174521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c02cbe0 con 0x7fa1180047c0 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.409+0000 7fa1325c9640 1 -- 172.21.15.64:0/3714174521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180047c0 msgr2=0x7fa118004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.409+0000 7fa1325c9640 1 --2- 172.21.15.64:0/3714174521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1180047c0 0x7fa118004c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009ac0 tx=0x7fa1180057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.409+0000 7fa131dc8640 1 -- 172.21.15.64:0/3714174521 reap_dead start 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.410+0000 7fa1325c9640 1 -- 172.21.15.64:0/3714174521 shutdown_connections 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.410+0000 7fa1325c9640 1 -- 172.21.15.64:0/3714174521 >> 172.21.15.64:0/3714174521 conn(0x7fa12c0d0e00 msgr2=0x7fa12c0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.410+0000 7fa1325c9640 1 -- 172.21.15.64:0/3714174521 shutdown_connections 2023-11-24T06:28:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.410+0000 7fa1325c9640 1 -- 172.21.15.64:0/3714174521 wait complete. 2023-11-24T06:28:44.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.412+0000 7fa1325c9640 1 Processor -- start 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.412+0000 7fa1325c9640 1 -- start start 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa1325c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180047c0 0x7fa12c152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa1325c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c1f2d40 0x7fa12c1f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa1325c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa12c0d1210 0x7fa12c0d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa1325c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c1f3f00 con 0x7fa12c0d1210 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa1325c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0d1bd0 con 0x7fa1180047c0 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa1325c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c152780 con 0x7fa12c1f2d40 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa130dc6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c1f2d40 0x7fa12c1f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa130dc6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c1f2d40 0x7fa12c1f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44164/0 (socket says 172.21.15.64:44164) 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa130dc6640 1 -- 172.21.15.64:0/3353347989 learned_addr learned my addr 172.21.15.64:0/3353347989 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.413+0000 7fa131dc8640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa12c0d1210 0x7fa12c0d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa1315c7640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180047c0 0x7fa12c152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa130dc6640 1 -- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180047c0 msgr2=0x7fa12c152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180047c0 0x7fa12c152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa130dc6640 1 -- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa12c0d1210 msgr2=0x7fa12c0d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa12c0d1210 0x7fa12c0d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa130dc6640 1 -- 172.21.15.64:0/3353347989 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c1f3870 con 0x7fa12c1f2d40 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa131dc8640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa12c0d1210 0x7fa12c0d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.414+0000 7fa130dc6640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c1f2d40 0x7fa12c1f31c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa11c037040 tx=0x7fa11c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.415+0000 7fa121ffb640 1 -- 172.21.15.64:0/3353347989 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003810 con 0x7fa12c1f2d40 2023-11-24T06:28:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.415+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c1f3b50 con 0x7fa12c1f2d40 2023-11-24T06:28:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.415+0000 7fa121ffb640 1 -- 172.21.15.64:0/3353347989 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c003970 con 0x7fa12c1f2d40 2023-11-24T06:28:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.415+0000 7fa121ffb640 1 -- 172.21.15.64:0/3353347989 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003d20 con 0x7fa12c1f2d40 2023-11-24T06:28:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.415+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c1f43f0 con 0x7fa12c1f2d40 2023-11-24T06:28:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.416+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c1f4580 con 0x7fa12c1f2d40 2023-11-24T06:28:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.418+0000 7fa121ffb640 1 -- 172.21.15.64:0/3353347989 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa11c040450 con 0x7fa12c1f2d40 2023-11-24T06:28:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.419+0000 7fa121ffb640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa11006ae70 0x7fa11006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.419+0000 7fa121ffb640 1 -- 172.21.15.64:0/3353347989 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fa11c0ab4e0 con 0x7fa12c1f2d40 2023-11-24T06:28:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.419+0000 7fa1315c7640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa11006ae70 0x7fa11006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.420+0000 7fa1315c7640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa11006ae70 0x7fa11006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa128001050 tx=0x7fa128009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.444+0000 7fa121ffb640 1 -- 172.21.15.64:0/3353347989 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa11c076540 con 0x7fa12c1f2d40 2023-11-24T06:28:44.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.676+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa12c067b60 con 0x7fa12c1f2d40 2023-11-24T06:28:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.677+0000 7fa121ffb640 1 -- 172.21.15.64:0/3353347989 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa11c079b60 con 0x7fa12c1f2d40 2023-11-24T06:28:44.680 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:44.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa11006ae70 msgr2=0x7fa11006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa11006ae70 0x7fa11006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa128001050 tx=0x7fa128009040 comp rx=0 tx=0).stop 2023-11-24T06:28:44.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c1f2d40 msgr2=0x7fa12c1f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c1f2d40 0x7fa12c1f31c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa11c037040 tx=0x7fa11c004730 comp rx=0 tx=0).stop 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 shutdown_connections 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa11006ae70 0x7fa11006d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa12c0d1210 0x7fa12c0d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa12c1f2d40 0x7fa12c1f31c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 --2- 172.21.15.64:0/3353347989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1180047c0 0x7fa12c152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 >> 172.21.15.64:0/3353347989 conn(0x7fa12c0d0e00 msgr2=0x7fa12c1606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 shutdown_connections 2023-11-24T06:28:44.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.679+0000 7fa1325c9640 1 -- 172.21.15.64:0/3353347989 wait complete. 2023-11-24T06:28:44.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:44.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.932+0000 7f6436e32640 1 Processor -- start 2023-11-24T06:28:44.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.933+0000 7f6436e32640 1 -- start start 2023-11-24T06:28:44.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.934+0000 7f6436e32640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64300adf20 0x7f6430070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.934+0000 7f6436e32640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643015f4e0 con 0x7f64300ccb50 2023-11-24T06:28:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.934+0000 7f6436e32640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6430071130 con 0x7f643015bf10 2023-11-24T06:28:44.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.934+0000 7f6436e32640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643015f970 con 0x7f64300adf20 2023-11-24T06:28:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6436631640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f64300ccb50 0x7f6430143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34824/0 (socket says 172.21.15.64:34824) 2023-11-24T06:28:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6436631640 1 -- 172.21.15.64:0/3643914789 learned_addr learned my addr 172.21.15.64:0/3643914789 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6435e30640 1 --2- 172.21.15.64:0/3643914789 >> v2:172.21.15.162:3300/0 conn(0x7f64300adf20 0x7f6430070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6434e2e640 1 -- 172.21.15.64:0/3643914789 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976599745 0 0) 0x7f643015f4e0 con 0x7f64300ccb50 2023-11-24T06:28:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6434e2e640 1 -- 172.21.15.64:0/3643914789 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f641c003520 con 0x7f64300ccb50 2023-11-24T06:28:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6434e2e640 1 -- 172.21.15.64:0/3643914789 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171177127 0 0) 0x7f6430071130 con 0x7f643015bf10 2023-11-24T06:28:44.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6434e2e640 1 -- 172.21.15.64:0/3643914789 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643015f4e0 con 0x7f643015bf10 2023-11-24T06:28:44.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6435e30640 1 -- 172.21.15.64:0/3643914789 >> v1:172.21.15.162:6789/0 conn(0x7f643015bf10 legacy=0x7f64300cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6435e30640 1 -- 172.21.15.64:0/3643914789 >> v1:172.21.15.64:6789/0 conn(0x7f64300ccb50 legacy=0x7f6430143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.935+0000 7f6435e30640 1 -- 172.21.15.64:0/3643914789 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643015c390 con 0x7f64300adf20 2023-11-24T06:28:44.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.937+0000 7f6435e30640 1 --2- 172.21.15.64:0/3643914789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 0x7f6430070a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f642c0097a0 tx=0x7f642c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f54e96fb9ffe24f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.937+0000 7f6434e2e640 1 -- 172.21.15.64:0/3643914789 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c02e070 con 0x7f64300adf20 2023-11-24T06:28:44.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.937+0000 7f6434e2e640 1 -- 172.21.15.64:0/3643914789 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c0315a0 con 0x7f64300adf20 2023-11-24T06:28:44.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.937+0000 7f6434e2e640 1 -- 172.21.15.64:0/3643914789 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c031910 con 0x7f64300adf20 2023-11-24T06:28:44.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.938+0000 7f6436e32640 1 -- 172.21.15.64:0/3643914789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 msgr2=0x7f6430070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.938+0000 7f6436e32640 1 --2- 172.21.15.64:0/3643914789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 0x7f6430070a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f642c0097a0 tx=0x7f642c02b840 comp rx=0 tx=0).stop 2023-11-24T06:28:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.938+0000 7f6436e32640 1 -- 172.21.15.64:0/3643914789 shutdown_connections 2023-11-24T06:28:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.938+0000 7f6436e32640 1 --2- 172.21.15.64:0/3643914789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 0x7f6430070a30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.938+0000 7f6436e32640 1 -- 172.21.15.64:0/3643914789 >> 172.21.15.64:0/3643914789 conn(0x7f643006a7d0 msgr2=0x7f643006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.938+0000 7f6436e32640 1 -- 172.21.15.64:0/3643914789 shutdown_connections 2023-11-24T06:28:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.938+0000 7f6436e32640 1 -- 172.21.15.64:0/3643914789 wait complete. 2023-11-24T06:28:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.940+0000 7f6436e32640 1 Processor -- start 2023-11-24T06:28:44.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.940+0000 7f6436e32640 1 -- start start 2023-11-24T06:28:44.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f6436e32640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 0x7f643015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f6436e32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64300ccb50 0x7f64300b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f6436e32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643015bf10 0x7f64300b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f6436e32640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643013b030 con 0x7f64300ccb50 2023-11-24T06:28:44.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f6436e32640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64301371b0 con 0x7f64300adf20 2023-11-24T06:28:44.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f6436e32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643015aee0 con 0x7f643015bf10 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f643562f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64300ccb50 0x7f64300b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f643562f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64300ccb50 0x7f64300b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33730/0 (socket says 172.21.15.64:33730) 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f643562f640 1 -- 172.21.15.64:0/319435929 learned_addr learned my addr 172.21.15.64:0/319435929 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.942+0000 7f6435e30640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 0x7f643015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.943+0000 7f643562f640 1 -- 172.21.15.64:0/319435929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643015bf10 msgr2=0x7f64300b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.943+0000 7f643562f640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643015bf10 0x7f64300b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.943+0000 7f643562f640 1 -- 172.21.15.64:0/319435929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 msgr2=0x7f643015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.943+0000 7f643562f640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 0x7f643015a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.943+0000 7f643562f640 1 -- 172.21.15.64:0/319435929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c009400 con 0x7f64300ccb50 2023-11-24T06:28:44.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.943+0000 7f643562f640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64300ccb50 0x7f64300b2ee0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f642000c410 tx=0x7f642000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.944+0000 7f64267fc640 1 -- 172.21.15.64:0/319435929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64200035f0 con 0x7f64300ccb50 2023-11-24T06:28:44.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.944+0000 7f64267fc640 1 -- 172.21.15.64:0/319435929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6420003750 con 0x7f64300ccb50 2023-11-24T06:28:44.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.944+0000 7f64267fc640 1 -- 172.21.15.64:0/319435929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420003b20 con 0x7f64300ccb50 2023-11-24T06:28:44.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.944+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64300cab00 con 0x7f64300ccb50 2023-11-24T06:28:44.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.944+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301376e0 con 0x7f64300ccb50 2023-11-24T06:28:44.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.945+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6430067450 con 0x7f64300ccb50 2023-11-24T06:28:44.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.947+0000 7f64267fc640 1 -- 172.21.15.64:0/319435929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6420003c80 con 0x7f64300ccb50 2023-11-24T06:28:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.948+0000 7f64267fc640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f641406ae40 0x7f641406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.948+0000 7f64267fc640 1 -- 172.21.15.64:0/319435929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f6420088b60 con 0x7f64300ccb50 2023-11-24T06:28:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.949+0000 7f6435e30640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f641406ae40 0x7f641406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.949+0000 7f6435e30640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f641406ae40 0x7f641406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f642c036000 tx=0x7f642c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:44.955+0000 7f64267fc640 1 -- 172.21.15.64:0/319435929 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6420053c50 con 0x7f64300ccb50 2023-11-24T06:28:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.214+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64300b0eb0 con 0x7f64300ccb50 2023-11-24T06:28:45.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.214+0000 7f64267fc640 1 -- 172.21.15.64:0/319435929 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6420017070 con 0x7f64300ccb50 2023-11-24T06:28:45.218 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:45.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.216+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f641406ae40 msgr2=0x7f641406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.216+0000 7f6436e32640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f641406ae40 0x7f641406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f642c036000 tx=0x7f642c008030 comp rx=0 tx=0).stop 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.216+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64300ccb50 msgr2=0x7f64300b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.216+0000 7f6436e32640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64300ccb50 0x7f64300b2ee0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f642000c410 tx=0x7f642000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 shutdown_connections 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f641406ae40 0x7f641406d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643015bf10 0x7f64300b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64300ccb50 0x7f64300b2ee0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 --2- 172.21.15.64:0/319435929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64300adf20 0x7f643015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 >> 172.21.15.64:0/319435929 conn(0x7f643006a7d0 msgr2=0x7f6430071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 shutdown_connections 2023-11-24T06:28:45.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.217+0000 7f6436e32640 1 -- 172.21.15.64:0/319435929 wait complete. 2023-11-24T06:28:45.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:45.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.436+0000 7f539bfff640 1 Processor -- start 2023-11-24T06:28:45.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.436+0000 7f539bfff640 1 -- start start 2023-11-24T06:28:45.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.438+0000 7f539bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.438+0000 7f539bfff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539c0d0a10 con 0x7f539c0cf9f0 2023-11-24T06:28:45.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.438+0000 7f539bfff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539c161e30 con 0x7f539c0cfe70 2023-11-24T06:28:45.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.438+0000 7f539bfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c0d1f50 con 0x7f539c160920 2023-11-24T06:28:45.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.439+0000 7f539affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.439+0000 7f539a7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f539c0cf9f0 0x7f539c161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34836/0 (socket says 172.21.15.64:34836) 2023-11-24T06:28:45.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.439+0000 7f539a7fc640 1 -- 172.21.15.64:0/3905872565 learned_addr learned my addr 172.21.15.64:0/3905872565 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:45.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.439+0000 7f5399ffb640 1 -- 172.21.15.64:0/3905872565 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819579054 0 0) 0x7f539c0d0a10 con 0x7f539c0cf9f0 2023-11-24T06:28:45.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.439+0000 7f5399ffb640 1 -- 172.21.15.64:0/3905872565 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5384003520 con 0x7f539c0cf9f0 2023-11-24T06:28:45.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.439+0000 7f539affd640 1 -- 172.21.15.64:0/3905872565 >> v1:172.21.15.162:6789/0 conn(0x7f539c0cfe70 legacy=0x7f539c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.440+0000 7f539affd640 1 -- 172.21.15.64:0/3905872565 >> v1:172.21.15.64:6789/0 conn(0x7f539c0cf9f0 legacy=0x7f539c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.440+0000 7f539affd640 1 -- 172.21.15.64:0/3905872565 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c0cf080 con 0x7f539c160920 2023-11-24T06:28:45.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.441+0000 7f539affd640 1 --2- 172.21.15.64:0/3905872565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c0b3f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f538c0097d0 tx=0x7f538c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea1356688ed235f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f5399ffb640 1 -- 172.21.15.64:0/3905872565 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c030610 con 0x7f539c160920 2023-11-24T06:28:45.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f5399ffb640 1 -- 172.21.15.64:0/3905872565 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c030770 con 0x7f539c160920 2023-11-24T06:28:45.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f5399ffb640 1 -- 172.21.15.64:0/3905872565 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c030b00 con 0x7f539c160920 2023-11-24T06:28:45.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f539bfff640 1 -- 172.21.15.64:0/3905872565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 msgr2=0x7f539c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f539bfff640 1 --2- 172.21.15.64:0/3905872565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c0b3f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f538c0097d0 tx=0x7f538c02b960 comp rx=0 tx=0).stop 2023-11-24T06:28:45.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f539bfff640 1 -- 172.21.15.64:0/3905872565 shutdown_connections 2023-11-24T06:28:45.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f539bfff640 1 --2- 172.21.15.64:0/3905872565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c0b3f30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.442+0000 7f539bfff640 1 -- 172.21.15.64:0/3905872565 >> 172.21.15.64:0/3905872565 conn(0x7f539c0d3090 msgr2=0x7f539c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.443+0000 7f539bfff640 1 -- 172.21.15.64:0/3905872565 shutdown_connections 2023-11-24T06:28:45.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.443+0000 7f539bfff640 1 -- 172.21.15.64:0/3905872565 wait complete. 2023-11-24T06:28:45.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.444+0000 7f539bfff640 1 Processor -- start 2023-11-24T06:28:45.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.444+0000 7f539bfff640 1 -- start start 2023-11-24T06:28:45.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.445+0000 7f539bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c0cf9f0 0x7f539c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.445+0000 7f539bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f539c0cfe70 0x7f539c153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.445+0000 7f539bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.445+0000 7f539bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c1f6610 con 0x7f539c0cf9f0 2023-11-24T06:28:45.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.445+0000 7f539bfff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c15fa30 con 0x7f539c0cfe70 2023-11-24T06:28:45.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.445+0000 7f539bfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c15fd10 con 0x7f539c160920 2023-11-24T06:28:45.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.446+0000 7f539a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f539c0cfe70 0x7f539c153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.446+0000 7f539affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c0cf9f0 0x7f539c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.446+0000 7f539a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f539c0cfe70 0x7f539c153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:47040/0 (socket says 172.21.15.64:47040) 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.446+0000 7f539a7fc640 1 -- 172.21.15.64:0/1423244567 learned_addr learned my addr 172.21.15.64:0/1423244567 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.446+0000 7f539b7fe640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.447+0000 7f539a7fc640 1 -- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 msgr2=0x7f539c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.447+0000 7f539a7fc640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.447+0000 7f539a7fc640 1 -- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c0cf9f0 msgr2=0x7f539c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.447+0000 7f539a7fc640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c0cf9f0 0x7f539c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.447+0000 7f539a7fc640 1 -- 172.21.15.64:0/1423244567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c009400 con 0x7f539c0cfe70 2023-11-24T06:28:45.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.447+0000 7f539affd640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c0cf9f0 0x7f539c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:45.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.447+0000 7f539a7fc640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f539c0cfe70 0x7f539c153850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f539000c410 tx=0x7f539000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.448+0000 7f5383fff640 1 -- 172.21.15.64:0/1423244567 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390005ce0 con 0x7f539c0cfe70 2023-11-24T06:28:45.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.448+0000 7f5383fff640 1 -- 172.21.15.64:0/1423244567 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5390005e40 con 0x7f539c0cfe70 2023-11-24T06:28:45.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.448+0000 7f5383fff640 1 -- 172.21.15.64:0/1423244567 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53900072f0 con 0x7f539c0cfe70 2023-11-24T06:28:45.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.448+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c15fff0 con 0x7f539c0cfe70 2023-11-24T06:28:45.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.448+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c153d90 con 0x7f539c0cfe70 2023-11-24T06:28:45.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.450+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c1f6aa0 con 0x7f539c0cfe70 2023-11-24T06:28:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.451+0000 7f5383fff640 1 -- 172.21.15.64:0/1423244567 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5390007510 con 0x7f539c0cfe70 2023-11-24T06:28:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.452+0000 7f5383fff640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f538806ae40 0x7f538806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.452+0000 7f5383fff640 1 -- 172.21.15.64:0/1423244567 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f5390088c50 con 0x7f539c0cfe70 2023-11-24T06:28:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.452+0000 7f539affd640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f538806ae40 0x7f538806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.453+0000 7f539affd640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f538806ae40 0x7f538806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f538c036000 tx=0x7f538c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.465+0000 7f5383fff640 1 -- 172.21.15.64:0/1423244567 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5390053e80 con 0x7f539c0cfe70 2023-11-24T06:28:45.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.743+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f539c1f6aa0 con 0x7f539c0cfe70 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.744+0000 7f5383fff640 1 -- 172.21.15.64:0/1423244567 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f539c1f6aa0 con 0x7f539c0cfe70 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:45.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.746+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f538806ae40 msgr2=0x7f538806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.746+0000 7f539bfff640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f538806ae40 0x7f538806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f538c036000 tx=0x7f538c008030 comp rx=0 tx=0).stop 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.746+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f539c0cfe70 msgr2=0x7f539c153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.746+0000 7f539bfff640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f539c0cfe70 0x7f539c153850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f539000c410 tx=0x7f539000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 shutdown_connections 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f538806ae40 0x7f538806d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f539c160920 0x7f539c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f539c0cfe70 0x7f539c153850 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 --2- 172.21.15.64:0/1423244567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c0cf9f0 0x7f539c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 >> 172.21.15.64:0/1423244567 conn(0x7f539c0d3090 msgr2=0x7f539c0c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 shutdown_connections 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:45.747+0000 7f539bfff640 1 -- 172.21.15.64:0/1423244567 wait complete. 2023-11-24T06:28:45.769 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T06:28:45.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:45.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-24T06:28:45.856 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:45.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:28:45.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:45.856 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:28:45.907 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:28:45.907 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:45.953 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:28:45.954 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:46.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:46.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.250+0000 7fb28a1f4640 1 Processor -- start 2023-11-24T06:28:46.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.250+0000 7fb28a1f4640 1 -- start start 2023-11-24T06:28:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.251+0000 7fb28a1f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.251+0000 7fb28a1f4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb2840cfa40 0x7fb2840cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.251+0000 7fb28a1f4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840b2c70 con 0x7fb2840cde90 2023-11-24T06:28:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.251+0000 7fb28a1f4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840d03c0 con 0x7fb2840cfa40 2023-11-24T06:28:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.251+0000 7fb28a1f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840cf6d0 con 0x7fb28415d3c0 2023-11-24T06:28:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2889f1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb2840cfa40 0x7fb2840cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2891f2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2889f1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb2840cfa40 0x7fb2840cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33750/0 (socket says 172.21.15.64:33750) 2023-11-24T06:28:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2889f1640 1 -- 172.21.15.64:0/4161035775 learned_addr learned my addr 172.21.15.64:0/4161035775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2889f1640 1 -- 172.21.15.64:0/4161035775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 msgr2=0x7fb2840d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2889f1640 1 --2- 172.21.15.64:0/4161035775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2889f1640 1 -- 172.21.15.64:0/4161035775 >> v1:172.21.15.162:6789/0 conn(0x7fb2840cde90 legacy=0x7fb2840ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2889f1640 1 -- 172.21.15.64:0/4161035775 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840cf3f0 con 0x7fb2840cfa40 2023-11-24T06:28:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.252+0000 7fb2891f2640 1 --2- 172.21.15.64:0/4161035775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.253+0000 7fb2889f1640 1 --2- 172.21.15.64:0/4161035775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cfe80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb274009870 tx=0x7fb27402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b4b79b0e342640 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.254+0000 7fb27bfff640 1 -- 172.21.15.64:0/4161035775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274030840 con 0x7fb2840cfa40 2023-11-24T06:28:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.254+0000 7fb27bfff640 1 -- 172.21.15.64:0/4161035775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2740309a0 con 0x7fb2840cfa40 2023-11-24T06:28:46.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.254+0000 7fb27bfff640 1 -- 172.21.15.64:0/4161035775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274030d30 con 0x7fb2840cfa40 2023-11-24T06:28:46.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.254+0000 7fb28a1f4640 1 -- 172.21.15.64:0/4161035775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 msgr2=0x7fb2840cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.255+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/4161035775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cfe80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb274009870 tx=0x7fb27402b980 comp rx=0 tx=0).stop 2023-11-24T06:28:46.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.255+0000 7fb28a1f4640 1 -- 172.21.15.64:0/4161035775 shutdown_connections 2023-11-24T06:28:46.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.255+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/4161035775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cfe80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.255+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/4161035775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.255+0000 7fb28a1f4640 1 -- 172.21.15.64:0/4161035775 >> 172.21.15.64:0/4161035775 conn(0x7fb2840d3050 msgr2=0x7fb2840d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:46.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.255+0000 7fb28a1f4640 1 -- 172.21.15.64:0/4161035775 shutdown_connections 2023-11-24T06:28:46.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.255+0000 7fb28a1f4640 1 -- 172.21.15.64:0/4161035775 wait complete. 2023-11-24T06:28:46.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.257+0000 7fb28a1f4640 1 Processor -- start 2023-11-24T06:28:46.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.258+0000 7fb28a1f4640 1 -- start start 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb28a1f4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2840cde90 0x7fb28415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb28a1f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb28a1f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb28a1f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840b7900 con 0x7fb2840cfa40 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb28a1f4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840a0ce0 con 0x7fb2840cde90 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb28a1f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840cc370 con 0x7fb28415d3c0 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb2889f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.259+0000 7fb2889f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33766/0 (socket says 172.21.15.64:33766) 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2889f1640 1 -- 172.21.15.64:0/214436880 learned_addr learned my addr 172.21.15.64:0/214436880 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2891f2640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2840cde90 0x7fb28415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2899f3640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2889f1640 1 -- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 msgr2=0x7fb2840a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2889f1640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2889f1640 1 -- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2840cde90 msgr2=0x7fb28415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2889f1640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2840cde90 0x7fb28415a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2889f1640 1 -- 172.21.15.64:0/214436880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740094d0 con 0x7fb2840cfa40 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.260+0000 7fb2889f1640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cbce0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb274000c00 tx=0x7fb274004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.261+0000 7fb279ffb640 1 -- 172.21.15.64:0/214436880 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274039c90 con 0x7fb2840cfa40 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.261+0000 7fb279ffb640 1 -- 172.21.15.64:0/214436880 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb274039df0 con 0x7fb2840cfa40 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.261+0000 7fb279ffb640 1 -- 172.21.15.64:0/214436880 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274043640 con 0x7fb2840cfa40 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.261+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2840b7010 con 0x7fb2840cfa40 2023-11-24T06:28:46.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.261+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2840b7590 con 0x7fb2840cfa40 2023-11-24T06:28:46.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.263+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2840b7ab0 con 0x7fb2840cfa40 2023-11-24T06:28:46.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.263+0000 7fb279ffb640 1 -- 172.21.15.64:0/214436880 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb2740437a0 con 0x7fb2840cfa40 2023-11-24T06:28:46.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.265+0000 7fb279ffb640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb26806ae40 0x7fb26806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.265+0000 7fb279ffb640 1 -- 172.21.15.64:0/214436880 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fb27403e070 con 0x7fb2840cfa40 2023-11-24T06:28:46.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.265+0000 7fb2891f2640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb26806ae40 0x7fb26806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.266+0000 7fb2891f2640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb26806ae40 0x7fb26806d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb280006e80 tx=0x7fb280008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.268+0000 7fb279ffb640 1 -- 172.21.15.64:0/214436880 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2840b7ab0 con 0x7fb2840cfa40 2023-11-24T06:28:46.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.522+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2840b7ab0 con 0x7fb2840cfa40 2023-11-24T06:28:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.524+0000 7fb279ffb640 1 -- 172.21.15.64:0/214436880 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7fb2840b7ab0 con 0x7fb2840cfa40 2023-11-24T06:28:46.528 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:46.528 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb26806ae40 msgr2=0x7fb26806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb26806ae40 0x7fb26806d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb280006e80 tx=0x7fb280008040 comp rx=0 tx=0).stop 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 msgr2=0x7fb2840cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cbce0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb274000c00 tx=0x7fb274004730 comp rx=0 tx=0).stop 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 shutdown_connections 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb26806ae40 0x7fb26806d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb28415d3c0 0x7fb2840a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2840cfa40 0x7fb2840cbce0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 --2- 172.21.15.64:0/214436880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2840cde90 0x7fb28415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 >> 172.21.15.64:0/214436880 conn(0x7fb2840d3050 msgr2=0x7fb2840b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:46.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 shutdown_connections 2023-11-24T06:28:46.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.528+0000 7fb28a1f4640 1 -- 172.21.15.64:0/214436880 wait complete. 2023-11-24T06:28:46.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:46.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.773+0000 7f89f1316640 1 Processor -- start 2023-11-24T06:28:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.774+0000 7f89f1316640 1 -- start start 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.775+0000 7f89f1316640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec15d4b0 0x7f89ec0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.775+0000 7f89f1316640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ec0b2730 con 0x7f89ec0cf9a0 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.775+0000 7f89f1316640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ec0d1ec0 con 0x7f89ec0b41e0 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.775+0000 7f89f1316640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0b29d0 con 0x7f89ec15d4b0 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89ebfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec15d4b0 0x7f89ec0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89eb7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f89ec0cf9a0 0x7f89ec0cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34850/0 (socket says 172.21.15.64:34850) 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89eb7fe640 1 -- 172.21.15.64:0/1855048349 learned_addr learned my addr 172.21.15.64:0/1855048349 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89eaffd640 1 -- 172.21.15.64:0/1855048349 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841625795 0 0) 0x7f89ec0b2730 con 0x7f89ec0cf9a0 2023-11-24T06:28:46.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89eaffd640 1 -- 172.21.15.64:0/1855048349 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d4003520 con 0x7f89ec0cf9a0 2023-11-24T06:28:46.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89ebfff640 1 -- 172.21.15.64:0/1855048349 >> v1:172.21.15.162:6789/0 conn(0x7f89ec0b41e0 legacy=0x7f89ec0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89ebfff640 1 -- 172.21.15.64:0/1855048349 >> v1:172.21.15.64:6789/0 conn(0x7f89ec0cf9a0 legacy=0x7f89ec0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.776+0000 7f89ebfff640 1 -- 172.21.15.64:0/1855048349 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec0b2cb0 con 0x7f89ec15d4b0 2023-11-24T06:28:46.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.778+0000 7f89ebfff640 1 --2- 172.21.15.64:0/1855048349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec15d4b0 0x7f89ec0d1550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009800 tx=0x7f89dc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58d3b40c02931cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.779+0000 7f89eaffd640 1 -- 172.21.15.64:0/1855048349 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc030800 con 0x7f89ec15d4b0 2023-11-24T06:28:46.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.779+0000 7f89eaffd640 1 -- 172.21.15.64:0/1855048349 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc030960 con 0x7f89ec15d4b0 2023-11-24T06:28:46.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.779+0000 7f89eaffd640 1 -- 172.21.15.64:0/1855048349 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc030cd0 con 0x7f89ec15d4b0 2023-11-24T06:28:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.779+0000 7f89f1316640 1 -- 172.21.15.64:0/1855048349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec15d4b0 msgr2=0x7f89ec0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.779+0000 7f89f1316640 1 --2- 172.21.15.64:0/1855048349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec15d4b0 0x7f89ec0d1550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009800 tx=0x7f89dc02b940 comp rx=0 tx=0).stop 2023-11-24T06:28:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.780+0000 7f89f1316640 1 -- 172.21.15.64:0/1855048349 shutdown_connections 2023-11-24T06:28:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.780+0000 7f89f1316640 1 --2- 172.21.15.64:0/1855048349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec15d4b0 0x7f89ec0d1550 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.780+0000 7f89f1316640 1 -- 172.21.15.64:0/1855048349 >> 172.21.15.64:0/1855048349 conn(0x7f89ec0d3000 msgr2=0x7f89ec0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.780+0000 7f89f1316640 1 -- 172.21.15.64:0/1855048349 shutdown_connections 2023-11-24T06:28:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.780+0000 7f89f1316640 1 -- 172.21.15.64:0/1855048349 wait complete. 2023-11-24T06:28:46.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.782+0000 7f89f1316640 1 Processor -- start 2023-11-24T06:28:46.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.782+0000 7f89f1316640 1 -- start start 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89f1316640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ec0b41e0 0x7f89ec0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89f1316640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec0cf9a0 0x7f89ec15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89f1316640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ec15d4b0 0x7f89ec0b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89ebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ec0b41e0 0x7f89ec0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89f1316640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0b8ba0 con 0x7f89ec0b41e0 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89f1316640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec15ee20 con 0x7f89ec15d4b0 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89f1316640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0b6fa0 con 0x7f89ec0cf9a0 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89ebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ec0b41e0 0x7f89ec0ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33780/0 (socket says 172.21.15.64:33780) 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89ebfff640 1 -- 172.21.15.64:0/261672829 learned_addr learned my addr 172.21.15.64:0/261672829 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:46.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89eb7fe640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec0cf9a0 0x7f89ec15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89ebfff640 1 -- 172.21.15.64:0/261672829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec0cf9a0 msgr2=0x7f89ec15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.784+0000 7f89ebfff640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec0cf9a0 0x7f89ec15e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89ebfff640 1 -- 172.21.15.64:0/261672829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ec15d4b0 msgr2=0x7f89ec0b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89ebfff640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ec15d4b0 0x7f89ec0b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89ebfff640 1 -- 172.21.15.64:0/261672829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc009400 con 0x7f89ec0b41e0 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89ebfff640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ec0b41e0 0x7f89ec0ca0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009720 tx=0x7f89dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89e8ff9640 1 -- 172.21.15.64:0/261672829 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc03f070 con 0x7f89ec0b41e0 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89e8ff9640 1 -- 172.21.15.64:0/261672829 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc003610 con 0x7f89ec0b41e0 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89e8ff9640 1 -- 172.21.15.64:0/261672829 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc003980 con 0x7f89ec0b41e0 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ec15da10 con 0x7f89ec0b41e0 2023-11-24T06:28:46.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.785+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ec0b69e0 con 0x7f89ec0b41e0 2023-11-24T06:28:46.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.787+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ec06cc10 con 0x7f89ec0b41e0 2023-11-24T06:28:46.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.787+0000 7f89e8ff9640 1 -- 172.21.15.64:0/261672829 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89dc003ae0 con 0x7f89ec0b41e0 2023-11-24T06:28:46.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.788+0000 7f89e8ff9640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f89d806ae40 0x7f89d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.788+0000 7f89e8ff9640 1 -- 172.21.15.64:0/261672829 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f89dc0ab4f0 con 0x7f89ec0b41e0 2023-11-24T06:28:46.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.789+0000 7f89eb7fe640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f89d806ae40 0x7f89d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.789+0000 7f89eb7fe640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f89d806ae40 0x7f89d806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f89ec1f67b0 tx=0x7f89e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:46.796+0000 7f89e8ff9640 1 -- 172.21.15.64:0/261672829 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89dc076550 con 0x7f89ec0b41e0 2023-11-24T06:28:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.044+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89ec067450 con 0x7f89ec0b41e0 2023-11-24T06:28:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.045+0000 7f89e8ff9640 1 -- 172.21.15.64:0/261672829 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89dc079b70 con 0x7f89ec0b41e0 2023-11-24T06:28:47.048 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.047+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f89d806ae40 msgr2=0x7f89d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.047+0000 7f89f1316640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f89d806ae40 0x7f89d806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f89ec1f67b0 tx=0x7f89e0009040 comp rx=0 tx=0).stop 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.047+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ec0b41e0 msgr2=0x7f89ec0ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.047+0000 7f89f1316640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ec0b41e0 0x7f89ec0ca0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009720 tx=0x7f89dc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 shutdown_connections 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f89d806ae40 0x7f89d806d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89ec15d4b0 0x7f89ec0b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ec0cf9a0 0x7f89ec15e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 --2- 172.21.15.64:0/261672829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89ec0b41e0 0x7f89ec0ca0d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 >> 172.21.15.64:0/261672829 conn(0x7f89ec0d3000 msgr2=0x7f89ec0cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 shutdown_connections 2023-11-24T06:28:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.048+0000 7f89f1316640 1 -- 172.21.15.64:0/261672829 wait complete. 2023-11-24T06:28:47.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:47.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.278+0000 7fb10bfff640 1 Processor -- start 2023-11-24T06:28:47.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.279+0000 7fb10bfff640 1 -- start start 2023-11-24T06:28:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c15d1d0 0x7fb10c0d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb10c0b1f70 0x7fb10c0d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10bfff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0d0090 con 0x7fb10c15d5b0 2023-11-24T06:28:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10bfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0d02e0 con 0x7fb10c0b1f70 2023-11-24T06:28:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10bfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0d05c0 con 0x7fb10c15d1d0 2023-11-24T06:28:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb10c0b1f70 0x7fb10c0d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c15d1d0 0x7fb10c0d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb10c0b1f70 0x7fb10c0d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33792/0 (socket says 172.21.15.64:33792) 2023-11-24T06:28:47.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10a7fc640 1 -- 172.21.15.64:0/2140211148 learned_addr learned my addr 172.21.15.64:0/2140211148 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:47.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.280+0000 7fb10b7fe640 1 --1- 172.21.15.64:0/2140211148 >> v1:172.21.15.162:6789/0 conn(0x7fb10c15d5b0 0x7fb10c0cdb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:37854/0 (socket says 172.21.15.64:37854) 2023-11-24T06:28:47.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.281+0000 7fb10a7fc640 1 -- 172.21.15.64:0/2140211148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c15d1d0 msgr2=0x7fb10c0d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.281+0000 7fb10a7fc640 1 --2- 172.21.15.64:0/2140211148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c15d1d0 0x7fb10c0d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.281+0000 7fb10a7fc640 1 -- 172.21.15.64:0/2140211148 >> v1:172.21.15.162:6789/0 conn(0x7fb10c15d5b0 legacy=0x7fb10c0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.281+0000 7fb10a7fc640 1 -- 172.21.15.64:0/2140211148 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0d08a0 con 0x7fb10c0b1f70 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.283+0000 7fb10a7fc640 1 --2- 172.21.15.64:0/2140211148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c0b1f70 0x7fb10c0d1b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb100009870 tx=0x7fb10002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238b99e6361bd63e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.283+0000 7fb109ffb640 1 -- 172.21.15.64:0/2140211148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100030840 con 0x7fb10c0b1f70 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.283+0000 7fb109ffb640 1 -- 172.21.15.64:0/2140211148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1000309a0 con 0x7fb10c0b1f70 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.283+0000 7fb109ffb640 1 -- 172.21.15.64:0/2140211148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100030d30 con 0x7fb10c0b1f70 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 -- 172.21.15.64:0/2140211148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c0b1f70 msgr2=0x7fb10c0d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2140211148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c0b1f70 0x7fb10c0d1b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb100009870 tx=0x7fb10002b980 comp rx=0 tx=0).stop 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 -- 172.21.15.64:0/2140211148 shutdown_connections 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2140211148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c0b1f70 0x7fb10c0d1b70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2140211148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c15d1d0 0x7fb10c0d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 -- 172.21.15.64:0/2140211148 >> 172.21.15.64:0/2140211148 conn(0x7fb10c0d3050 msgr2=0x7fb10c0d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 -- 172.21.15.64:0/2140211148 shutdown_connections 2023-11-24T06:28:47.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.284+0000 7fb10bfff640 1 -- 172.21.15.64:0/2140211148 wait complete. 2023-11-24T06:28:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.286+0000 7fb10bfff640 1 Processor -- start 2023-11-24T06:28:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.287+0000 7fb10bfff640 1 -- start start 2023-11-24T06:28:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b1f70 0x7fb10c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c15d1d0 0x7fb10c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c15d5b0 0x7fb10c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0b6670 con 0x7fb10c15d1d0 2023-11-24T06:28:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10bfff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0cc730 con 0x7fb10c15d5b0 2023-11-24T06:28:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10bfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0b6090 con 0x7fb10c0b1f70 2023-11-24T06:28:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b1f70 0x7fb10c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b1f70 0x7fb10c1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44216/0 (socket says 172.21.15.64:44216) 2023-11-24T06:28:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10affd640 1 -- 172.21.15.64:0/2014871447 learned_addr learned my addr 172.21.15.64:0/2014871447 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10a7fc640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c15d1d0 0x7fb10c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.288+0000 7fb10b7fe640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c15d5b0 0x7fb10c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10affd640 1 -- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c15d5b0 msgr2=0x7fb10c0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10affd640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c15d5b0 0x7fb10c0cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10affd640 1 -- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c15d1d0 msgr2=0x7fb10c0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10affd640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c15d1d0 0x7fb10c0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10affd640 1 -- 172.21.15.64:0/2014871447 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000094d0 con 0x7fb10c0b1f70 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10b7fe640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c15d5b0 0x7fb10c0cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10a7fc640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c15d1d0 0x7fb10c0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.289+0000 7fb10affd640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b1f70 0x7fb10c1590e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00e8a0 tx=0x7fb0fc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.290+0000 7fb0f3fff640 1 -- 172.21.15.64:0/2014871447 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0094f0 con 0x7fb10c0b1f70 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.290+0000 7fb0f3fff640 1 -- 172.21.15.64:0/2014871447 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00c940 con 0x7fb10c0b1f70 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.290+0000 7fb0f3fff640 1 -- 172.21.15.64:0/2014871447 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00ccf0 con 0x7fb10c0b1f70 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.290+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c160070 con 0x7fb10c0b1f70 2023-11-24T06:28:47.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.290+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c159620 con 0x7fb10c0b1f70 2023-11-24T06:28:47.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.291+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c067450 con 0x7fb10c0b1f70 2023-11-24T06:28:47.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.293+0000 7fb0f3fff640 1 -- 172.21.15.64:0/2014871447 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0fc013470 con 0x7fb10c0b1f70 2023-11-24T06:28:47.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.295+0000 7fb0f3fff640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb0f806ae40 0x7fb0f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.295+0000 7fb0f3fff640 1 -- 172.21.15.64:0/2014871447 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fb0fc089e90 con 0x7fb10c0b1f70 2023-11-24T06:28:47.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.295+0000 7fb10a7fc640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb0f806ae40 0x7fb0f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.296+0000 7fb10a7fc640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb0f806ae40 0x7fb0f806d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb100036000 tx=0x7fb100008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.301+0000 7fb0f3fff640 1 -- 172.21.15.64:0/2014871447 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0fc054f70 con 0x7fb10c0b1f70 2023-11-24T06:28:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.565+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb10c0b46f0 con 0x7fb10c0b1f70 2023-11-24T06:28:47.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.566+0000 7fb0f3fff640 1 -- 172.21.15.64:0/2014871447 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0fc017020 con 0x7fb10c0b1f70 2023-11-24T06:28:47.568 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb0f806ae40 msgr2=0x7fb0f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb0f806ae40 0x7fb0f806d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb100036000 tx=0x7fb100008030 comp rx=0 tx=0).stop 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b1f70 msgr2=0x7fb10c1590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b1f70 0x7fb10c1590e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00e8a0 tx=0x7fb0fc00ed60 comp rx=0 tx=0).stop 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 shutdown_connections 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb0f806ae40 0x7fb0f806d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c15d5b0 0x7fb10c0cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c15d1d0 0x7fb10c0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 --2- 172.21.15.64:0/2014871447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b1f70 0x7fb10c1590e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 >> 172.21.15.64:0/2014871447 conn(0x7fb10c0d3050 msgr2=0x7fb10c0d38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 shutdown_connections 2023-11-24T06:28:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.569+0000 7fb10bfff640 1 -- 172.21.15.64:0/2014871447 wait complete. 2023-11-24T06:28:47.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.767+0000 7fd4c1143640 1 Processor -- start 2023-11-24T06:28:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.767+0000 7fd4c1143640 1 -- start start 2023-11-24T06:28:47.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4c1143640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4bc15d4b0 0x7fd4bc0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4c1143640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4c1143640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4bc0d0320 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4c1143640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc0b2730 con 0x7fd4bc15d4b0 2023-11-24T06:28:47.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4c1143640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0d1ec0 con 0x7fd4bc0b41e0 2023-11-24T06:28:47.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4bb7fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd4bc0cf9a0 0x7fd4bc0cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:37864/0 (socket says 172.21.15.64:37864) 2023-11-24T06:28:47.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4bb7fe640 1 -- 172.21.15.64:0/1311884793 learned_addr learned my addr 172.21.15.64:0/1311884793 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:47.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4c0942640 1 --2- 172.21.15.64:0/1311884793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.769+0000 7fd4bbfff640 1 --2- 172.21.15.64:0/1311884793 >> v2:172.21.15.162:3300/0 conn(0x7fd4bc15d4b0 0x7fd4bc0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.770+0000 7fd4c0942640 1 -- 172.21.15.64:0/1311884793 >> v2:172.21.15.162:3300/0 conn(0x7fd4bc15d4b0 msgr2=0x7fd4bc0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.770+0000 7fd4c0942640 1 --2- 172.21.15.64:0/1311884793 >> v2:172.21.15.162:3300/0 conn(0x7fd4bc15d4b0 0x7fd4bc0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.770+0000 7fd4c0942640 1 -- 172.21.15.64:0/1311884793 >> v1:172.21.15.162:6789/0 conn(0x7fd4bc0cf9a0 legacy=0x7fd4bc0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.770+0000 7fd4c0942640 1 -- 172.21.15.64:0/1311884793 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0b29d0 con 0x7fd4bc0b41e0 2023-11-24T06:28:47.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.771+0000 7fd4c0942640 1 --2- 172.21.15.64:0/1311884793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc0cfde0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009d90 tx=0x7fd4b402bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f25fe749231ded6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.772+0000 7fd4baffd640 1 -- 172.21.15.64:0/1311884793 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b402ca10 con 0x7fd4bc0b41e0 2023-11-24T06:28:47.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.772+0000 7fd4baffd640 1 -- 172.21.15.64:0/1311884793 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b402cb70 con 0x7fd4bc0b41e0 2023-11-24T06:28:47.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.772+0000 7fd4baffd640 1 -- 172.21.15.64:0/1311884793 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4034440 con 0x7fd4bc0b41e0 2023-11-24T06:28:47.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 -- 172.21.15.64:0/1311884793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 msgr2=0x7fd4bc0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 --2- 172.21.15.64:0/1311884793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc0cfde0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009d90 tx=0x7fd4b402bce0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 -- 172.21.15.64:0/1311884793 shutdown_connections 2023-11-24T06:28:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 --2- 172.21.15.64:0/1311884793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc0cfde0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 --2- 172.21.15.64:0/1311884793 >> v2:172.21.15.162:3300/0 conn(0x7fd4bc15d4b0 0x7fd4bc0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 -- 172.21.15.64:0/1311884793 >> 172.21.15.64:0/1311884793 conn(0x7fd4bc0d3000 msgr2=0x7fd4bc0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 -- 172.21.15.64:0/1311884793 shutdown_connections 2023-11-24T06:28:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.773+0000 7fd4c1143640 1 -- 172.21.15.64:0/1311884793 wait complete. 2023-11-24T06:28:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.775+0000 7fd4c1143640 1 Processor -- start 2023-11-24T06:28:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.775+0000 7fd4c1143640 1 -- start start 2023-11-24T06:28:47.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.777+0000 7fd4c1143640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc09d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.777+0000 7fd4c1143640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4bc0cf9a0 0x7fd4bc15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.777+0000 7fd4c1143640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4bc15d4b0 0x7fd4bc09eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.777+0000 7fd4c1143640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0b5d50 con 0x7fd4bc15d4b0 2023-11-24T06:28:47.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.777+0000 7fd4c1143640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc09e7a0 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.777+0000 7fd4c1143640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc09df00 con 0x7fd4bc0b41e0 2023-11-24T06:28:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4bc0cf9a0 0x7fd4bc15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4bc0cf9a0 0x7fd4bc15c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:47078/0 (socket says 172.21.15.64:47078) 2023-11-24T06:28:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 -- 172.21.15.64:0/4089047236 learned_addr learned my addr 172.21.15.64:0/4089047236 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bbfff640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc09d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4c0942640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4bc15d4b0 0x7fd4bc09eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 -- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 msgr2=0x7fd4bc09d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc09d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 -- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4bc15d4b0 msgr2=0x7fd4bc09eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4bc15d4b0 0x7fd4bc09eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.778+0000 7fd4bb7fe640 1 -- 172.21.15.64:0/4089047236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b4009a70 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.779+0000 7fd4bb7fe640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4bc0cf9a0 0x7fd4bc15c580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c800 tx=0x7fd4b000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.779+0000 7fd4b8ff9640 1 -- 172.21.15.64:0/4089047236 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00072d0 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.779+0000 7fd4b8ff9640 1 -- 172.21.15.64:0/4089047236 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b0007430 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.779+0000 7fd4b8ff9640 1 -- 172.21.15.64:0/4089047236 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00047e0 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.779+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc15da10 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.779+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc09e4c0 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.781+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc067450 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.781+0000 7fd4b8ff9640 1 -- 172.21.15.64:0/4089047236 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4b0004960 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.783+0000 7fd4b8ff9640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd4a806ae40 0x7fd4a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.783+0000 7fd4b8ff9640 1 -- 172.21.15.64:0/4089047236 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fd4b0088ae0 con 0x7fd4bc0cf9a0 2023-11-24T06:28:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.783+0000 7fd4bbfff640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd4a806ae40 0x7fd4a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.784+0000 7fd4bbfff640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd4a806ae40 0x7fd4a806d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009aa0 tx=0x7fd4ac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:47.791+0000 7fd4b8ff9640 1 -- 172.21.15.64:0/4089047236 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4b0053b40 con 0x7fd4bc0cf9a0 2023-11-24T06:28:48.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.082+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4bc09dbe0 con 0x7fd4bc0cf9a0 2023-11-24T06:28:48.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.083+0000 7fd4b8ff9640 1 -- 172.21.15.64:0/4089047236 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4b0018070 con 0x7fd4bc0cf9a0 2023-11-24T06:28:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:48.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.085+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd4a806ae40 msgr2=0x7fd4a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.085+0000 7fd4c1143640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd4a806ae40 0x7fd4a806d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009aa0 tx=0x7fd4ac009190 comp rx=0 tx=0).stop 2023-11-24T06:28:48.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.085+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4bc0cf9a0 msgr2=0x7fd4bc15c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.085+0000 7fd4c1143640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4bc0cf9a0 0x7fd4bc15c580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c800 tx=0x7fd4b000ccc0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 shutdown_connections 2023-11-24T06:28:48.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd4a806ae40 0x7fd4a806d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4bc15d4b0 0x7fd4bc09eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4bc0cf9a0 0x7fd4bc15c580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 --2- 172.21.15.64:0/4089047236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4bc0b41e0 0x7fd4bc09d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 >> 172.21.15.64:0/4089047236 conn(0x7fd4bc0d3000 msgr2=0x7fd4bc0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 shutdown_connections 2023-11-24T06:28:48.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.086+0000 7fd4c1143640 1 -- 172.21.15.64:0/4089047236 wait complete. 2023-11-24T06:28:48.111 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:48.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.326+0000 7fbb46c2d640 1 Processor -- start 2023-11-24T06:28:48.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.326+0000 7fbb46c2d640 1 -- start start 2023-11-24T06:28:48.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb46c2d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb4013df80 0x7fbb4015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb46c2d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb400cf9a0 0x7fbb400cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb46c2d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4006c470 con 0x7fbb4013e360 2023-11-24T06:28:48.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb46c2d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4006c750 con 0x7fbb4013df80 2023-11-24T06:28:48.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb46c2d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4006ca30 con 0x7fbb400cf9a0 2023-11-24T06:28:48.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb3ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbb4013e360 0x7fbb400cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34862/0 (socket says 172.21.15.64:34862) 2023-11-24T06:28:48.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb449c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb4013df80 0x7fbb4015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb3ffff640 1 -- 172.21.15.64:0/4281827932 learned_addr learned my addr 172.21.15.64:0/4281827932 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:48.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.328+0000 7fbb451ca640 1 --2- 172.21.15.64:0/4281827932 >> v2:172.21.15.162:3300/0 conn(0x7fbb400cf9a0 0x7fbb400cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.329+0000 7fbb449c9640 1 -- 172.21.15.64:0/4281827932 >> v2:172.21.15.162:3300/0 conn(0x7fbb400cf9a0 msgr2=0x7fbb400cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:48.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.329+0000 7fbb449c9640 1 --2- 172.21.15.64:0/4281827932 >> v2:172.21.15.162:3300/0 conn(0x7fbb400cf9a0 0x7fbb400cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.329+0000 7fbb449c9640 1 -- 172.21.15.64:0/4281827932 >> v1:172.21.15.64:6789/0 conn(0x7fbb4013e360 legacy=0x7fbb400cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.329+0000 7fbb449c9640 1 -- 172.21.15.64:0/4281827932 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4006cd10 con 0x7fbb4013df80 2023-11-24T06:28:48.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.330+0000 7fbb449c9640 1 --2- 172.21.15.64:0/4281827932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013df80 0x7fbb4015bee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb400ccb10 tx=0x7fbb3402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a0e07c3d63b413 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.330+0000 7fbb3f7fe640 1 -- 172.21.15.64:0/4281827932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34030710 con 0x7fbb4013df80 2023-11-24T06:28:48.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb3f7fe640 1 -- 172.21.15.64:0/4281827932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3402e050 con 0x7fbb4013df80 2023-11-24T06:28:48.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb3f7fe640 1 -- 172.21.15.64:0/4281827932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34030a90 con 0x7fbb4013df80 2023-11-24T06:28:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb46c2d640 1 -- 172.21.15.64:0/4281827932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013df80 msgr2=0x7fbb4015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/4281827932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013df80 0x7fbb4015bee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb400ccb10 tx=0x7fbb3402b710 comp rx=0 tx=0).stop 2023-11-24T06:28:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb46c2d640 1 -- 172.21.15.64:0/4281827932 shutdown_connections 2023-11-24T06:28:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/4281827932 >> v2:172.21.15.162:3300/0 conn(0x7fbb400cf9a0 0x7fbb400cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/4281827932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013df80 0x7fbb4015bee0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb46c2d640 1 -- 172.21.15.64:0/4281827932 >> 172.21.15.64:0/4281827932 conn(0x7fbb4006a800 msgr2=0x7fbb4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.331+0000 7fbb46c2d640 1 -- 172.21.15.64:0/4281827932 shutdown_connections 2023-11-24T06:28:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.332+0000 7fbb46c2d640 1 -- 172.21.15.64:0/4281827932 wait complete. 2023-11-24T06:28:48.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.333+0000 7fbb46c2d640 1 Processor -- start 2023-11-24T06:28:48.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.334+0000 7fbb46c2d640 1 -- start start 2023-11-24T06:28:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.334+0000 7fbb46c2d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400cf9a0 0x7fbb40071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb46c2d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb4013df80 0x7fbb40147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb46c2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013e360 0x7fbb40147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb46c2d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb401469f0 con 0x7fbb4013e360 2023-11-24T06:28:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb46c2d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40146e80 con 0x7fbb4013df80 2023-11-24T06:28:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb46c2d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40145de0 con 0x7fbb400cf9a0 2023-11-24T06:28:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb451ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013e360 0x7fbb40147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb451ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013e360 0x7fbb40147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33822/0 (socket says 172.21.15.64:33822) 2023-11-24T06:28:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb451ca640 1 -- 172.21.15.64:0/2737682210 learned_addr learned my addr 172.21.15.64:0/2737682210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb3ffff640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb4013df80 0x7fbb40147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.335+0000 7fbb449c9640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400cf9a0 0x7fbb40071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.336+0000 7fbb3ffff640 1 -- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400cf9a0 msgr2=0x7fbb40071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.336+0000 7fbb3ffff640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400cf9a0 0x7fbb40071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.336+0000 7fbb3ffff640 1 -- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013e360 msgr2=0x7fbb40147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.336+0000 7fbb3ffff640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013e360 0x7fbb40147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.336+0000 7fbb3ffff640 1 -- 172.21.15.64:0/2737682210 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34009400 con 0x7fbb4013df80 2023-11-24T06:28:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.336+0000 7fbb3ffff640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb4013df80 0x7fbb40147100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb400cc790 tx=0x7fbb2c00cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.336+0000 7fbb3d7fa640 1 -- 172.21.15.64:0/2737682210 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0072d0 con 0x7fbb4013df80 2023-11-24T06:28:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.337+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb401460c0 con 0x7fbb4013df80 2023-11-24T06:28:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.337+0000 7fbb3d7fa640 1 -- 172.21.15.64:0/2737682210 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c007430 con 0x7fbb4013df80 2023-11-24T06:28:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.337+0000 7fbb3d7fa640 1 -- 172.21.15.64:0/2737682210 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0047e0 con 0x7fbb4013df80 2023-11-24T06:28:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.337+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb40146680 con 0x7fbb4013df80 2023-11-24T06:28:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.338+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb40146840 con 0x7fbb4013df80 2023-11-24T06:28:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.339+0000 7fbb3d7fa640 1 -- 172.21.15.64:0/2737682210 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb2c004940 con 0x7fbb4013df80 2023-11-24T06:28:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.341+0000 7fbb3d7fa640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbb2406ae40 0x7fbb2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.341+0000 7fbb3d7fa640 1 -- 172.21.15.64:0/2737682210 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fbb2c088a90 con 0x7fbb4013df80 2023-11-24T06:28:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.342+0000 7fbb449c9640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbb2406ae40 0x7fbb2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.343+0000 7fbb449c9640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbb2406ae40 0x7fbb2406d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbb34034000 tx=0x7fbb34008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.349+0000 7fbb3d7fa640 1 -- 172.21.15.64:0/2737682210 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb2c053a60 con 0x7fbb4013df80 2023-11-24T06:28:48.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.629+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbb400675d0 con 0x7fbb4013df80 2023-11-24T06:28:48.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.630+0000 7fbb3d7fa640 1 -- 172.21.15.64:0/2737682210 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbb2c018030 con 0x7fbb4013df80 2023-11-24T06:28:48.633 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbb2406ae40 msgr2=0x7fbb2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbb2406ae40 0x7fbb2406d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbb34034000 tx=0x7fbb34008030 comp rx=0 tx=0).stop 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb4013df80 msgr2=0x7fbb40147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb4013df80 0x7fbb40147100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb400cc790 tx=0x7fbb2c00cbe0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 shutdown_connections 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbb2406ae40 0x7fbb2406d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb4013e360 0x7fbb40147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb4013df80 0x7fbb40147100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 --2- 172.21.15.64:0/2737682210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400cf9a0 0x7fbb40071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 >> 172.21.15.64:0/2737682210 conn(0x7fbb4006a800 msgr2=0x7fbb4015d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 shutdown_connections 2023-11-24T06:28:48.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.633+0000 7fbb46c2d640 1 -- 172.21.15.64:0/2737682210 wait complete. 2023-11-24T06:28:48.652 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:28:48.652 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:48.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:48.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.916+0000 7fc64b59e640 1 Processor -- start 2023-11-24T06:28:48.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.916+0000 7fc64b59e640 1 -- start start 2023-11-24T06:28:48.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.917+0000 7fc64b59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc64c143c40 0x7fc64c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.917+0000 7fc64b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c070e50 con 0x7fc64c0717d0 2023-11-24T06:28:48.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.917+0000 7fc64b59e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c144020 con 0x7fc64c0ccc10 2023-11-24T06:28:48.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.917+0000 7fc64b59e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c0723c0 con 0x7fc64c143c40 2023-11-24T06:28:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.917+0000 7fc649d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc64c0717d0 0x7fc64c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34870/0 (socket says 172.21.15.64:34870) 2023-11-24T06:28:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.917+0000 7fc649d9b640 1 -- 172.21.15.64:0/2508441827 learned_addr learned my addr 172.21.15.64:0/2508441827 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64a59c640 1 --2- 172.21.15.64:0/2508441827 >> v2:172.21.15.162:3300/0 conn(0x7fc64c143c40 0x7fc64c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045190671 0 0) 0x7fc64c070e50 con 0x7fc64c0717d0 2023-11-24T06:28:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc634003520 con 0x7fc64c0717d0 2023-11-24T06:28:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3936902126 0 0) 0x7fc634003520 con 0x7fc64c0717d0 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 >> v2:172.21.15.162:3300/0 conn(0x7fc64c143c40 msgr2=0x7fc64c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 --2- 172.21.15.64:0/2508441827 >> v2:172.21.15.162:3300/0 conn(0x7fc64c143c40 0x7fc64c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 >> v1:172.21.15.162:6789/0 conn(0x7fc64c0ccc10 legacy=0x7fc64c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c071e00 con 0x7fc64c0717d0 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fc640001670 con 0x7fc64c0717d0 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 >> v1:172.21.15.64:6789/0 conn(0x7fc64c0717d0 legacy=0x7fc64c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.918+0000 7fc64959a640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634003970 0x7fc634003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.919+0000 7fc64959a640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6340047c0 0x7fc634004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.919+0000 7fc64959a640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc634005180 0x7fc63400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.919+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0723c0 con 0x7fc634005180 2023-11-24T06:28:48.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.919+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63400c080 con 0x7fc634003970 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.919+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63400c240 con 0x7fc6340047c0 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.919+0000 7fc64ad9d640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc634005180 0x7fc63400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.919+0000 7fc649d9b640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6340047c0 0x7fc634004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.920+0000 7fc64a59c640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634003970 0x7fc634003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.920+0000 7fc64ad9d640 1 -- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6340047c0 msgr2=0x7fc634004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.920+0000 7fc64ad9d640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6340047c0 0x7fc634004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.920+0000 7fc64ad9d640 1 -- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634003970 msgr2=0x7fc634003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.920+0000 7fc64ad9d640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634003970 0x7fc634003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.920+0000 7fc64ad9d640 1 -- 172.21.15.64:0/2508441827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c071e00 con 0x7fc634005180 2023-11-24T06:28:48.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.921+0000 7fc64ad9d640 1 -- 172.21.15.64:0/2508441827 reap_dead start 2023-11-24T06:28:48.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.921+0000 7fc64ad9d640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc634005180 0x7fc63400b980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc644001e50 tx=0x7fc6340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d3e3c274ec2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.921+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644003b70 con 0x7fc634005180 2023-11-24T06:28:48.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.921+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644003d10 con 0x7fc634005180 2023-11-24T06:28:48.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.921+0000 7fc64959a640 1 -- 172.21.15.64:0/2508441827 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644005310 con 0x7fc634005180 2023-11-24T06:28:48.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.922+0000 7fc64b59e640 1 -- 172.21.15.64:0/2508441827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc634005180 msgr2=0x7fc63400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.922+0000 7fc64b59e640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc634005180 0x7fc63400b980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc644001e50 tx=0x7fc6340057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.922+0000 7fc64b59e640 1 -- 172.21.15.64:0/2508441827 shutdown_connections 2023-11-24T06:28:48.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.922+0000 7fc64b59e640 1 --2- 172.21.15.64:0/2508441827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc634005180 0x7fc63400b980 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.922+0000 7fc64b59e640 1 -- 172.21.15.64:0/2508441827 >> 172.21.15.64:0/2508441827 conn(0x7fc64c06a800 msgr2=0x7fc64c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.922+0000 7fc64b59e640 1 -- 172.21.15.64:0/2508441827 shutdown_connections 2023-11-24T06:28:48.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.922+0000 7fc64b59e640 1 -- 172.21.15.64:0/2508441827 wait complete. 2023-11-24T06:28:48.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.925+0000 7fc64b59e640 1 Processor -- start 2023-11-24T06:28:48.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.925+0000 7fc64b59e640 1 -- start start 2023-11-24T06:28:48.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc64b59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634005180 0x7fc64c139190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc64b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc64c15a6b0 0x7fc64c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc64b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64c159b60 0x7fc64c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc64b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c158ac0 con 0x7fc64c159b60 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc64b59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c158da0 con 0x7fc634005180 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc64b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c159080 con 0x7fc64c15a6b0 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc649d9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc64c15a6b0 0x7fc64c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc64ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64c159b60 0x7fc64c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc649d9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc64c15a6b0 0x7fc64c1401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:44264/0 (socket says 172.21.15.64:44264) 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.927+0000 7fc649d9b640 1 -- 172.21.15.64:0/1115585334 learned_addr learned my addr 172.21.15.64:0/1115585334 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.928+0000 7fc64a59c640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634005180 0x7fc64c139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.928+0000 7fc649d9b640 1 -- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634005180 msgr2=0x7fc64c139190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.928+0000 7fc649d9b640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634005180 0x7fc64c139190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.928+0000 7fc649d9b640 1 -- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64c159b60 msgr2=0x7fc64c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.928+0000 7fc649d9b640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64c159b60 0x7fc64c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.928+0000 7fc649d9b640 1 -- 172.21.15.64:0/1115585334 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c159360 con 0x7fc64c15a6b0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.928+0000 7fc649d9b640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc64c15a6b0 0x7fc64c1401e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6400072c0 tx=0x7fc640007780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.929+0000 7fc63affd640 1 -- 172.21.15.64:0/1115585334 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000fa00 con 0x7fc64c15a6b0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.929+0000 7fc63affd640 1 -- 172.21.15.64:0/1115585334 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000fb80 con 0x7fc64c15a6b0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.929+0000 7fc63affd640 1 -- 172.21.15.64:0/1115585334 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400133f0 con 0x7fc64c15a6b0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.929+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c140720 con 0x7fc64c15a6b0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.929+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c15d100 con 0x7fc64c15a6b0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.930+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c140940 con 0x7fc64c15a6b0 2023-11-24T06:28:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.933+0000 7fc63affd640 1 -- 172.21.15.64:0/1115585334 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc640013550 con 0x7fc64c15a6b0 2023-11-24T06:28:48.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.933+0000 7fc63affd640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc62c06ae40 0x7fc62c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.934+0000 7fc63affd640 1 -- 172.21.15.64:0/1115585334 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fc64008a2b0 con 0x7fc64c15a6b0 2023-11-24T06:28:48.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.934+0000 7fc64a59c640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc62c06ae40 0x7fc62c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.936+0000 7fc64a59c640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc62c06ae40 0x7fc62c06d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009910 tx=0x7fc63c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:48.938+0000 7fc63affd640 1 -- 172.21.15.64:0/1115585334 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc640055470 con 0x7fc64c15a6b0 2023-11-24T06:28:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.201+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc64c1594f0 con 0x7fc64c15a6b0 2023-11-24T06:28:49.206 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.203+0000 7fc63affd640 1 -- 172.21.15.64:0/1115585334 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7fc640058a90 con 0x7fc64c15a6b0 2023-11-24T06:28:49.208 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc62c06ae40 msgr2=0x7fc62c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc62c06ae40 0x7fc62c06d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009910 tx=0x7fc63c008040 comp rx=0 tx=0).stop 2023-11-24T06:28:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc64c15a6b0 msgr2=0x7fc64c1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc64c15a6b0 0x7fc64c1401e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6400072c0 tx=0x7fc640007780 comp rx=0 tx=0).stop 2023-11-24T06:28:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 shutdown_connections 2023-11-24T06:28:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc62c06ae40 0x7fc62c06d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64c159b60 0x7fc64c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc64c15a6b0 0x7fc64c1401e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 --2- 172.21.15.64:0/1115585334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc634005180 0x7fc64c139190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 >> 172.21.15.64:0/1115585334 conn(0x7fc64c06a800 msgr2=0x7fc64c0cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:49.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.206+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 shutdown_connections 2023-11-24T06:28:49.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.207+0000 7fc64b59e640 1 -- 172.21.15.64:0/1115585334 wait complete. 2023-11-24T06:28:49.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:49.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:49.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.459+0000 7fd72759e640 1 Processor -- start 2023-11-24T06:28:49.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.459+0000 7fd72759e640 1 -- start start 2023-11-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.460+0000 7fd72759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 0x7fd7280d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.460+0000 7fd72759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280b2730 con 0x7fd7280cf9a0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.460+0000 7fd72759e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280d1ec0 con 0x7fd7280b41e0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.460+0000 7fd72759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280b29d0 con 0x7fd72815d4b0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.461+0000 7fd725d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd7280cf9a0 0x7fd7280cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34878/0 (socket says 172.21.15.64:34878) 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.461+0000 7fd725d9b640 1 -- 172.21.15.64:0/4205426837 learned_addr learned my addr 172.21.15.64:0/4205426837 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.461+0000 7fd72659c640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 0x7fd7280d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.461+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336682389 0 0) 0x7fd7280b2730 con 0x7fd7280cf9a0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.461+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710003520 con 0x7fd7280cf9a0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87962344 0 0) 0x7fd7280d1ec0 con 0x7fd7280b41e0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7280b2730 con 0x7fd7280b41e0 2023-11-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014385183 0 0) 0x7fd710003520 con 0x7fd7280cf9a0 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 msgr2=0x7fd7280d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 0x7fd7280d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 >> v1:172.21.15.162:6789/0 conn(0x7fd7280b41e0 legacy=0x7fd7280ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280b2cb0 con 0x7fd7280cf9a0 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fd71c001670 con 0x7fd7280cf9a0 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 >> v1:172.21.15.64:6789/0 conn(0x7fd7280cf9a0 legacy=0x7fd7280cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 0x7fd710003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7100040e0 0x7fd7100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710004d00 0x7fd710005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280b29d0 con 0x7fd710004d00 2023-11-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000bb70 con 0x7fd7100040e0 2023-11-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000bd30 con 0x7fd72815d4b0 2023-11-24T06:28:49.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd726d9d640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710004d00 0x7fd710005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.462+0000 7fd72659c640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 0x7fd710003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.463+0000 7fd726d9d640 1 -- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 msgr2=0x7fd710003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.463+0000 7fd726d9d640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72815d4b0 0x7fd710003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.463+0000 7fd726d9d640 1 -- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7100040e0 msgr2=0x7fd7100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.463+0000 7fd726d9d640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7100040e0 0x7fd7100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.463+0000 7fd726d9d640 1 -- 172.21.15.64:0/4205426837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710003520 con 0x7fd710004d00 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.464+0000 7fd726d9d640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710004d00 0x7fd710005180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd720001fa0 tx=0x7fd7100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411ed0c997c0c024 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.465+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720003b70 con 0x7fd710004d00 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.465+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd720003cd0 con 0x7fd710004d00 2023-11-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.465+0000 7fd72559a640 1 -- 172.21.15.64:0/4205426837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200052f0 con 0x7fd710004d00 2023-11-24T06:28:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.466+0000 7fd72759e640 1 -- 172.21.15.64:0/4205426837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710004d00 msgr2=0x7fd710005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.466+0000 7fd72759e640 1 --2- 172.21.15.64:0/4205426837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710004d00 0x7fd710005180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd720001fa0 tx=0x7fd7100057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.466+0000 7fd726d9d640 1 -- 172.21.15.64:0/4205426837 reap_dead start 2023-11-24T06:28:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.466+0000 7fd72759e640 1 -- 172.21.15.64:0/4205426837 shutdown_connections 2023-11-24T06:28:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.466+0000 7fd72759e640 1 -- 172.21.15.64:0/4205426837 >> 172.21.15.64:0/4205426837 conn(0x7fd7280d3000 msgr2=0x7fd7280d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.466+0000 7fd72759e640 1 -- 172.21.15.64:0/4205426837 shutdown_connections 2023-11-24T06:28:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.466+0000 7fd72759e640 1 -- 172.21.15.64:0/4205426837 wait complete. 2023-11-24T06:28:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.468+0000 7fd72759e640 1 Processor -- start 2023-11-24T06:28:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.469+0000 7fd72759e640 1 -- start start 2023-11-24T06:28:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.470+0000 7fd72759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd710004d00 0x7fd7281f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.470+0000 7fd72759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281592e0 0x7fd728158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd72759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7281e8840 0x7fd72806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd72759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7281f3bc0 con 0x7fd7281e8840 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd72759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280c7dd0 con 0x7fd7281592e0 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd72759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280c80b0 con 0x7fd710004d00 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7281e8840 0x7fd72806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7281e8840 0x7fd72806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33842/0 (socket says 172.21.15.64:33842) 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 -- 172.21.15.64:0/3919907074 learned_addr learned my addr 172.21.15.64:0/3919907074 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd72659c640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd710004d00 0x7fd7281f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 -- 172.21.15.64:0/3919907074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd710004d00 msgr2=0x7fd7281f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd710004d00 0x7fd7281f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 -- 172.21.15.64:0/3919907074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281592e0 msgr2=0x7fd728158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281592e0 0x7fd728158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.471+0000 7fd726d9d640 1 -- 172.21.15.64:0/3919907074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72815da10 con 0x7fd7281e8840 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.472+0000 7fd726d9d640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7281e8840 0x7fd72806ae20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd720001fa0 tx=0x7fd720001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.472+0000 7fd716ffd640 1 -- 172.21.15.64:0/3919907074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720005580 con 0x7fd7281e8840 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.472+0000 7fd716ffd640 1 -- 172.21.15.64:0/3919907074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7200056e0 con 0x7fd7281e8840 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.472+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7280c8390 con 0x7fd7281e8840 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.472+0000 7fd716ffd640 1 -- 172.21.15.64:0/3919907074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72002c740 con 0x7fd7281e8840 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.472+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7280c8950 con 0x7fd7281e8840 2023-11-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.474+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7280c8b40 con 0x7fd7281e8840 2023-11-24T06:28:49.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.476+0000 7fd716ffd640 1 -- 172.21.15.64:0/3919907074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd72002c8a0 con 0x7fd7281e8840 2023-11-24T06:28:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.477+0000 7fd716ffd640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd70806ae40 0x7fd70806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.477+0000 7fd716ffd640 1 -- 172.21.15.64:0/3919907074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fd7200aa4a0 con 0x7fd7281e8840 2023-11-24T06:28:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.477+0000 7fd72659c640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd70806ae40 0x7fd70806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.478+0000 7fd72659c640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd70806ae40 0x7fd70806d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.483+0000 7fd716ffd640 1 -- 172.21.15.64:0/3919907074 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd720075580 con 0x7fd7281e8840 2023-11-24T06:28:49.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.770+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7280c8b40 con 0x7fd7281e8840 2023-11-24T06:28:49.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.771+0000 7fd716ffd640 1 -- 172.21.15.64:0/3919907074 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd720078ba0 con 0x7fd7281e8840 2023-11-24T06:28:49.774 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd70806ae40 msgr2=0x7fd70806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd70806ae40 0x7fd70806d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718008040 comp rx=0 tx=0).stop 2023-11-24T06:28:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7281e8840 msgr2=0x7fd72806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7281e8840 0x7fd72806ae20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd720001fa0 tx=0x7fd720001650 comp rx=0 tx=0).stop 2023-11-24T06:28:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 shutdown_connections 2023-11-24T06:28:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd70806ae40 0x7fd70806d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7281e8840 0x7fd72806ae20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281592e0 0x7fd728158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 --2- 172.21.15.64:0/3919907074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd710004d00 0x7fd7281f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 >> 172.21.15.64:0/3919907074 conn(0x7fd7280d3000 msgr2=0x7fd72815c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:49.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 shutdown_connections 2023-11-24T06:28:49.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:49.774+0000 7fd72759e640 1 -- 172.21.15.64:0/3919907074 wait complete. 2023-11-24T06:28:49.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:50.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.036+0000 7fb4b880e640 1 Processor -- start 2023-11-24T06:28:50.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.036+0000 7fb4b880e640 1 -- start start 2023-11-24T06:28:50.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.037+0000 7fb4b880e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4b4143ae0 0x7fb4b40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.037+0000 7fb4b880e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb4b40cf7d0 0x7fb4b415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.037+0000 7fb4b880e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b40cfc30 con 0x7fb4b415e1e0 2023-11-24T06:28:50.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.037+0000 7fb4b880e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b40b1ee0 con 0x7fb4b40cf7d0 2023-11-24T06:28:50.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.037+0000 7fb4b880e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b40b2780 con 0x7fb4b4143ae0 2023-11-24T06:28:50.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b2d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb4b415e1e0 0x7fb4b415a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34888/0 (socket says 172.21.15.64:34888) 2023-11-24T06:28:50.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b2d9d640 1 -- 172.21.15.64:0/299162232 learned_addr learned my addr 172.21.15.64:0/299162232 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:50.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b1d9b640 1 --2- 172.21.15.64:0/299162232 >> v2:172.21.15.64:3300/0 conn(0x7fb4b40cf7d0 0x7fb4b415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b259c640 1 --2- 172.21.15.64:0/299162232 >> v2:172.21.15.162:3300/0 conn(0x7fb4b4143ae0 0x7fb4b40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b159a640 1 -- 172.21.15.64:0/299162232 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043387052 0 0) 0x7fb4b40cfc30 con 0x7fb4b415e1e0 2023-11-24T06:28:50.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b159a640 1 -- 172.21.15.64:0/299162232 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb49c005730 con 0x7fb4b415e1e0 2023-11-24T06:28:50.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b1d9b640 1 -- 172.21.15.64:0/299162232 >> v2:172.21.15.162:3300/0 conn(0x7fb4b4143ae0 msgr2=0x7fb4b40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:50.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b1d9b640 1 --2- 172.21.15.64:0/299162232 >> v2:172.21.15.162:3300/0 conn(0x7fb4b4143ae0 0x7fb4b40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b1d9b640 1 -- 172.21.15.64:0/299162232 >> v1:172.21.15.64:6789/0 conn(0x7fb4b415e1e0 legacy=0x7fb4b415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.038+0000 7fb4b1d9b640 1 -- 172.21.15.64:0/299162232 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b415e990 con 0x7fb4b40cf7d0 2023-11-24T06:28:50.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.040+0000 7fb4b1d9b640 1 --2- 172.21.15.64:0/299162232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b40cf7d0 0x7fb4b415dca0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40cfea0 tx=0x7fb4a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67a188539da2f75 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.040+0000 7fb4b159a640 1 -- 172.21.15.64:0/299162232 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8031830 con 0x7fb4b40cf7d0 2023-11-24T06:28:50.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.040+0000 7fb4b159a640 1 -- 172.21.15.64:0/299162232 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8031990 con 0x7fb4b40cf7d0 2023-11-24T06:28:50.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.040+0000 7fb4b159a640 1 -- 172.21.15.64:0/299162232 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8031d00 con 0x7fb4b40cf7d0 2023-11-24T06:28:50.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 -- 172.21.15.64:0/299162232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b40cf7d0 msgr2=0x7fb4b415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 --2- 172.21.15.64:0/299162232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b40cf7d0 0x7fb4b415dca0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40cfea0 tx=0x7fb4a802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 -- 172.21.15.64:0/299162232 shutdown_connections 2023-11-24T06:28:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 --2- 172.21.15.64:0/299162232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b40cf7d0 0x7fb4b415dca0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 --2- 172.21.15.64:0/299162232 >> v2:172.21.15.162:3300/0 conn(0x7fb4b4143ae0 0x7fb4b40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 -- 172.21.15.64:0/299162232 >> 172.21.15.64:0/299162232 conn(0x7fb4b406a800 msgr2=0x7fb4b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 -- 172.21.15.64:0/299162232 shutdown_connections 2023-11-24T06:28:50.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.041+0000 7fb4b880e640 1 -- 172.21.15.64:0/299162232 wait complete. 2023-11-24T06:28:50.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.043+0000 7fb4b880e640 1 Processor -- start 2023-11-24T06:28:50.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.043+0000 7fb4b880e640 1 -- start start 2023-11-24T06:28:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.044+0000 7fb4b880e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4b40cf7d0 0x7fb4b40a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.044+0000 7fb4b880e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4b4143ae0 0x7fb4b41597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b880e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b415e1e0 0x7fb4b40aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b880e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40acb00 con 0x7fb4b415e1e0 2023-11-24T06:28:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b880e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40acda0 con 0x7fb4b40cf7d0 2023-11-24T06:28:50.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b880e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40ad080 con 0x7fb4b4143ae0 2023-11-24T06:28:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b415e1e0 0x7fb4b40aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b415e1e0 0x7fb4b40aaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33870/0 (socket says 172.21.15.64:33870) 2023-11-24T06:28:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b2d9d640 1 -- 172.21.15.64:0/4125141435 learned_addr learned my addr 172.21.15.64:0/4125141435 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b1d9b640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4b4143ae0 0x7fb4b41597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.045+0000 7fb4b259c640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4b40cf7d0 0x7fb4b40a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b2d9d640 1 -- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4b4143ae0 msgr2=0x7fb4b41597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b2d9d640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4b4143ae0 0x7fb4b41597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b2d9d640 1 -- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4b40cf7d0 msgr2=0x7fb4b40a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b2d9d640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4b40cf7d0 0x7fb4b40a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b2d9d640 1 -- 172.21.15.64:0/4125141435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8009400 con 0x7fb4b415e1e0 2023-11-24T06:28:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b1d9b640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4b4143ae0 0x7fb4b41597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b2d9d640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b415e1e0 0x7fb4b40aaa90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00cb40 tx=0x7fb4ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4a2ffd640 1 -- 172.21.15.64:0/4125141435 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00aca0 con 0x7fb4b415e1e0 2023-11-24T06:28:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4a2ffd640 1 -- 172.21.15.64:0/4125141435 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac00ae00 con 0x7fb4b415e1e0 2023-11-24T06:28:50.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b40ad360 con 0x7fb4b415e1e0 2023-11-24T06:28:50.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.046+0000 7fb4a2ffd640 1 -- 172.21.15.64:0/4125141435 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac012620 con 0x7fb4b415e1e0 2023-11-24T06:28:50.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.047+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b4159d10 con 0x7fb4b415e1e0 2023-11-24T06:28:50.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.048+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b40ad880 con 0x7fb4b415e1e0 2023-11-24T06:28:50.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.049+0000 7fb4a2ffd640 1 -- 172.21.15.64:0/4125141435 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4ac007070 con 0x7fb4b415e1e0 2023-11-24T06:28:50.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.052+0000 7fb4a2ffd640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb49406ae40 0x7fb49406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.052+0000 7fb4a2ffd640 1 -- 172.21.15.64:0/4125141435 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fb4ac0896b0 con 0x7fb4b415e1e0 2023-11-24T06:28:50.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.052+0000 7fb4b259c640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb49406ae40 0x7fb49406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.053+0000 7fb4b259c640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb49406ae40 0x7fb49406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009b40 tx=0x7fb4a4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.056+0000 7fb4a2ffd640 1 -- 172.21.15.64:0/4125141435 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4b40ad880 con 0x7fb4b415e1e0 2023-11-24T06:28:50.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.327+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4b40ad880 con 0x7fb4b415e1e0 2023-11-24T06:28:50.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.328+0000 7fb4a2ffd640 1 -- 172.21.15.64:0/4125141435 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4b40ad880 con 0x7fb4b415e1e0 2023-11-24T06:28:50.331 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb49406ae40 msgr2=0x7fb49406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb49406ae40 0x7fb49406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009b40 tx=0x7fb4a4009190 comp rx=0 tx=0).stop 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b415e1e0 msgr2=0x7fb4b40aaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b415e1e0 0x7fb4b40aaa90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00cb40 tx=0x7fb4ac00a060 comp rx=0 tx=0).stop 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 shutdown_connections 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb49406ae40 0x7fb49406d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4b415e1e0 0x7fb4b40aaa90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4b4143ae0 0x7fb4b41597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 --2- 172.21.15.64:0/4125141435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4b40cf7d0 0x7fb4b40a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 >> 172.21.15.64:0/4125141435 conn(0x7fb4b406a800 msgr2=0x7fb4b415b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.332+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 shutdown_connections 2023-11-24T06:28:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.333+0000 7fb4b880e640 1 -- 172.21.15.64:0/4125141435 wait complete. 2023-11-24T06:28:50.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.577+0000 7f6a8f59e640 1 Processor -- start 2023-11-24T06:28:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.577+0000 7f6a8f59e640 1 -- start start 2023-11-24T06:28:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.579+0000 7f6a8f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a900cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8f59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6a900b20d0 0x7f6a900b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8f59e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a900ce760 con 0x7f6a9015d580 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8f59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a900d0a00 con 0x7f6a900b20d0 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8f59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a900cf250 con 0x7f6a9015d960 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8e59c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6a9015d580 0x7f6a900cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:37902/0 (socket says 172.21.15.64:37902) 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8e59c640 1 -- 172.21.15.64:0/2004675933 learned_addr learned my addr 172.21.15.64:0/2004675933 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8ed9d640 1 --2- 172.21.15.64:0/2004675933 >> v2:172.21.15.64:3300/0 conn(0x7f6a900b20d0 0x7f6a900b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.580+0000 7f6a8dd9b640 1 --2- 172.21.15.64:0/2004675933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a900cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.581+0000 7f6a8ed9d640 1 -- 172.21.15.64:0/2004675933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 msgr2=0x7f6a900cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.581+0000 7f6a8ed9d640 1 --2- 172.21.15.64:0/2004675933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a900cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.581+0000 7f6a8ed9d640 1 -- 172.21.15.64:0/2004675933 >> v1:172.21.15.162:6789/0 conn(0x7f6a9015d580 legacy=0x7f6a900cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.581+0000 7f6a8ed9d640 1 -- 172.21.15.64:0/2004675933 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900d1f50 con 0x7f6a900b20d0 2023-11-24T06:28:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.582+0000 7f6a8ed9d640 1 --2- 172.21.15.64:0/2004675933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a900b20d0 0x7f6a900b0000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009d90 tx=0x7f6a8802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae7b28bd75ab180 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.583+0000 7f6a8d59a640 1 -- 172.21.15.64:0/2004675933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8802cbd0 con 0x7f6a900b20d0 2023-11-24T06:28:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.583+0000 7f6a8d59a640 1 -- 172.21.15.64:0/2004675933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8802cd30 con 0x7f6a900b20d0 2023-11-24T06:28:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.583+0000 7f6a8d59a640 1 -- 172.21.15.64:0/2004675933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8802cea0 con 0x7f6a900b20d0 2023-11-24T06:28:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2004675933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a900b20d0 msgr2=0x7f6a900b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2004675933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a900b20d0 0x7f6a900b0000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009d90 tx=0x7f6a8802be20 comp rx=0 tx=0).stop 2023-11-24T06:28:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2004675933 shutdown_connections 2023-11-24T06:28:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2004675933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a900b20d0 0x7f6a900b0000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2004675933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a900cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2004675933 >> 172.21.15.64:0/2004675933 conn(0x7f6a900d3040 msgr2=0x7f6a900d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2004675933 shutdown_connections 2023-11-24T06:28:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.584+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2004675933 wait complete. 2023-11-24T06:28:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.586+0000 7f6a8f59e640 1 Processor -- start 2023-11-24T06:28:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.586+0000 7f6a8f59e640 1 -- start start 2023-11-24T06:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.587+0000 7f6a8f59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a900b20d0 0x7f6a90159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.587+0000 7f6a8f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a9015d580 0x7f6a9015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.587+0000 7f6a8f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a9015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.587+0000 7f6a8f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a90159e80 con 0x7f6a9015d580 2023-11-24T06:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.587+0000 7f6a8f59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a900cb6e0 con 0x7f6a900b20d0 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.587+0000 7f6a8f59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9015ef30 con 0x7f6a9015d960 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.587+0000 7f6a8dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a9015d580 0x7f6a9015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8e59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a900b20d0 0x7f6a90159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a9015d580 0x7f6a9015a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33890/0 (socket says 172.21.15.64:33890) 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8dd9b640 1 -- 172.21.15.64:0/2065571299 learned_addr learned my addr 172.21.15.64:0/2065571299 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8ed9d640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a9015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8e59c640 1 -- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 msgr2=0x7f6a9015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8e59c640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a9015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8e59c640 1 -- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a9015d580 msgr2=0x7f6a9015a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8e59c640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a9015d580 0x7f6a9015a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8e59c640 1 -- 172.21.15.64:0/2065571299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a88009a70 con 0x7f6a900b20d0 2023-11-24T06:28:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.588+0000 7f6a8dd9b640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a9015d580 0x7f6a9015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.589+0000 7f6a8e59c640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a900b20d0 0x7f6a90159940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c440 tx=0x7f6a80001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.589+0000 7f6a7effd640 1 -- 172.21.15.64:0/2065571299 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80005ce0 con 0x7f6a900b20d0 2023-11-24T06:28:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.589+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a900cae40 con 0x7f6a900b20d0 2023-11-24T06:28:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.589+0000 7f6a7effd640 1 -- 172.21.15.64:0/2065571299 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80005e40 con 0x7f6a900b20d0 2023-11-24T06:28:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.589+0000 7f6a7effd640 1 -- 172.21.15.64:0/2065571299 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800072d0 con 0x7f6a900b20d0 2023-11-24T06:28:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.589+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a900cb400 con 0x7f6a900b20d0 2023-11-24T06:28:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.591+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a900b5a30 con 0x7f6a900b20d0 2023-11-24T06:28:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.592+0000 7f6a7effd640 1 -- 172.21.15.64:0/2065571299 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a80007430 con 0x7f6a900b20d0 2023-11-24T06:28:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.593+0000 7f6a7effd640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a7006ae40 0x7f6a7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.593+0000 7f6a7effd640 1 -- 172.21.15.64:0/2065571299 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f6a80088be0 con 0x7f6a900b20d0 2023-11-24T06:28:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.593+0000 7f6a8dd9b640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a7006ae40 0x7f6a7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.594+0000 7f6a8dd9b640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a7006ae40 0x7f6a7006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a900b12b0 tx=0x7f6a84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.610+0000 7f6a7effd640 1 -- 172.21.15.64:0/2065571299 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a80053bc0 con 0x7f6a900b20d0 2023-11-24T06:28:50.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.885+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a9006cd30 con 0x7f6a900b20d0 2023-11-24T06:28:50.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.886+0000 7f6a7effd640 1 -- 172.21.15.64:0/2065571299 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a800571e0 con 0x7f6a900b20d0 2023-11-24T06:28:50.889 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a7006ae40 msgr2=0x7f6a7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a7006ae40 0x7f6a7006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a900b12b0 tx=0x7f6a84008040 comp rx=0 tx=0).stop 2023-11-24T06:28:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a900b20d0 msgr2=0x7f6a90159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a900b20d0 0x7f6a90159940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c440 tx=0x7f6a80001e70 comp rx=0 tx=0).stop 2023-11-24T06:28:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 shutdown_connections 2023-11-24T06:28:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a7006ae40 0x7f6a7006d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a9015d960 0x7f6a9015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a9015d580 0x7f6a9015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 --2- 172.21.15.64:0/2065571299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a900b20d0 0x7f6a90159940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 >> 172.21.15.64:0/2065571299 conn(0x7f6a900d3040 msgr2=0x7f6a9015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 shutdown_connections 2023-11-24T06:28:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:50.889+0000 7f6a8f59e640 1 -- 172.21.15.64:0/2065571299 wait complete. 2023-11-24T06:28:50.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:51.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.118+0000 7f7f48fcb640 1 Processor -- start 2023-11-24T06:28:51.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.119+0000 7f7f48fcb640 1 -- start start 2023-11-24T06:28:51.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.120+0000 7f7f48fcb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f44160920 0x7f7f440b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.120+0000 7f7f48fcb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7f440cfe70 0x7f7f44161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.121+0000 7f7f48fcb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f440d0a10 con 0x7f7f440cf9f0 2023-11-24T06:28:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.121+0000 7f7f48fcb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f440d1f50 con 0x7f7f440cfe70 2023-11-24T06:28:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.121+0000 7f7f48fcb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440cf080 con 0x7f7f44160920 2023-11-24T06:28:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.121+0000 7f7f42ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7f440cf9f0 0x7f7f44161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34898/0 (socket says 172.21.15.64:34898) 2023-11-24T06:28:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.121+0000 7f7f42ffd640 1 -- 172.21.15.64:0/3849634440 learned_addr learned my addr 172.21.15.64:0/3849634440 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.121+0000 7f7f43fff640 1 --2- 172.21.15.64:0/3849634440 >> v2:172.21.15.162:3300/0 conn(0x7f7f440cfe70 0x7f7f44161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.121+0000 7f7f437fe640 1 --2- 172.21.15.64:0/3849634440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f44160920 0x7f7f440b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.122+0000 7f7f437fe640 1 -- 172.21.15.64:0/3849634440 >> v2:172.21.15.162:3300/0 conn(0x7f7f440cfe70 msgr2=0x7f7f44161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.122+0000 7f7f437fe640 1 --2- 172.21.15.64:0/3849634440 >> v2:172.21.15.162:3300/0 conn(0x7f7f440cfe70 0x7f7f44161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.122+0000 7f7f437fe640 1 -- 172.21.15.64:0/3849634440 >> v1:172.21.15.64:6789/0 conn(0x7f7f440cf9f0 legacy=0x7f7f44161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.122+0000 7f7f437fe640 1 -- 172.21.15.64:0/3849634440 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440c6be0 con 0x7f7f44160920 2023-11-24T06:28:51.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.124+0000 7f7f437fe640 1 --2- 172.21.15.64:0/3849634440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f44160920 0x7f7f440b3f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f340097a0 tx=0x7f7f3402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c7a8b9e70dfdcc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.129+0000 7f7f427fc640 1 -- 172.21.15.64:0/3849634440 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3402d040 con 0x7f7f44160920 2023-11-24T06:28:51.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.129+0000 7f7f427fc640 1 -- 172.21.15.64:0/3849634440 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f34031810 con 0x7f7f44160920 2023-11-24T06:28:51.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.129+0000 7f7f427fc640 1 -- 172.21.15.64:0/3849634440 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34031ba0 con 0x7f7f44160920 2023-11-24T06:28:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.129+0000 7f7f48fcb640 1 -- 172.21.15.64:0/3849634440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f44160920 msgr2=0x7f7f440b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.129+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/3849634440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f44160920 0x7f7f440b3f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f340097a0 tx=0x7f7f3402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.130+0000 7f7f48fcb640 1 -- 172.21.15.64:0/3849634440 shutdown_connections 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.130+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/3849634440 >> v2:172.21.15.162:3300/0 conn(0x7f7f440cfe70 0x7f7f44161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.130+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/3849634440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f44160920 0x7f7f440b3f30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.130+0000 7f7f48fcb640 1 -- 172.21.15.64:0/3849634440 >> 172.21.15.64:0/3849634440 conn(0x7f7f440d3090 msgr2=0x7f7f440d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.130+0000 7f7f48fcb640 1 -- 172.21.15.64:0/3849634440 shutdown_connections 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.130+0000 7f7f48fcb640 1 -- 172.21.15.64:0/3849634440 wait complete. 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.132+0000 7f7f48fcb640 1 Processor -- start 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.132+0000 7f7f48fcb640 1 -- start start 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f48fcb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f440cf9f0 0x7f7f4415d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f48fcb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f440cfe70 0x7f7f440caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f48fcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f44160920 0x7f7f4415f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f48fcb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440cb460 con 0x7f7f44160920 2023-11-24T06:28:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f48fcb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4415f760 con 0x7f7f440cfe70 2023-11-24T06:28:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f48fcb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440cb690 con 0x7f7f440cf9f0 2023-11-24T06:28:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f44160920 0x7f7f4415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f44160920 0x7f7f4415f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33904/0 (socket says 172.21.15.64:33904) 2023-11-24T06:28:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.133+0000 7f7f43fff640 1 -- 172.21.15.64:0/900893631 learned_addr learned my addr 172.21.15.64:0/900893631 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.134+0000 7f7f42ffd640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f440cfe70 0x7f7f440caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.134+0000 7f7f437fe640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f440cf9f0 0x7f7f4415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.134+0000 7f7f43fff640 1 -- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f440cf9f0 msgr2=0x7f7f4415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.134+0000 7f7f43fff640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f440cf9f0 0x7f7f4415d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.134+0000 7f7f43fff640 1 -- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f440cfe70 msgr2=0x7f7f440caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.134+0000 7f7f43fff640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f440cfe70 0x7f7f440caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.134+0000 7f7f43fff640 1 -- 172.21.15.64:0/900893631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34009400 con 0x7f7f44160920 2023-11-24T06:28:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.135+0000 7f7f43fff640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f44160920 0x7f7f4415f220 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00efc0 tx=0x7f7f3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.135+0000 7f7f2bfff640 1 -- 172.21.15.64:0/900893631 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c014480 con 0x7f7f44160920 2023-11-24T06:28:51.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.135+0000 7f7f2bfff640 1 -- 172.21.15.64:0/900893631 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00f040 con 0x7f7f44160920 2023-11-24T06:28:51.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.135+0000 7f7f2bfff640 1 -- 172.21.15.64:0/900893631 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c014820 con 0x7f7f44160920 2023-11-24T06:28:51.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.135+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f440cb970 con 0x7f7f44160920 2023-11-24T06:28:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.135+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f440cbf30 con 0x7f7f44160920 2023-11-24T06:28:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.137+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f44067450 con 0x7f7f44160920 2023-11-24T06:28:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.138+0000 7f7f2bfff640 1 -- 172.21.15.64:0/900893631 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f3c014980 con 0x7f7f44160920 2023-11-24T06:28:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.139+0000 7f7f2bfff640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7f3006ae40 0x7f7f3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.139+0000 7f7f2bfff640 1 -- 172.21.15.64:0/900893631 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f7f3c08a4c0 con 0x7f7f44160920 2023-11-24T06:28:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.140+0000 7f7f437fe640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7f3006ae40 0x7f7f3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.140+0000 7f7f437fe640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7f3006ae40 0x7f7f3006d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f34036000 tx=0x7f7f3403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.145+0000 7f7f2bfff640 1 -- 172.21.15.64:0/900893631 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f3c055490 con 0x7f7f44160920 2023-11-24T06:28:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.391+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7f440c69c0 con 0x7f7f44160920 2023-11-24T06:28:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.392+0000 7f7f2bfff640 1 -- 172.21.15.64:0/900893631 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7f3c018080 con 0x7f7f44160920 2023-11-24T06:28:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:28:51.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.394+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7f3006ae40 msgr2=0x7f7f3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.394+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7f3006ae40 0x7f7f3006d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f34036000 tx=0x7f7f3403c040 comp rx=0 tx=0).stop 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.394+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f44160920 msgr2=0x7f7f4415f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.394+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f44160920 0x7f7f4415f220 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00efc0 tx=0x7f7f3c00c490 comp rx=0 tx=0).stop 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 shutdown_connections 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7f3006ae40 0x7f7f3006d300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f44160920 0x7f7f4415f220 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f440cfe70 0x7f7f440caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 --2- 172.21.15.64:0/900893631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f440cf9f0 0x7f7f4415d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 >> 172.21.15.64:0/900893631 conn(0x7f7f440d3090 msgr2=0x7f7f4415af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:51.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 shutdown_connections 2023-11-24T06:28:51.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.395+0000 7f7f48fcb640 1 -- 172.21.15.64:0/900893631 wait complete. 2023-11-24T06:28:51.414 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2023-11-24T06:28:51.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:51.415 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:51.514 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:51.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:28:51.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:51.515 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:28:51.558 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:28:51.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.789+0000 7f41c6b67640 1 Processor -- start 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.789+0000 7f41c6b67640 1 -- start start 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c6b67640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c0143c40 0x7f41c0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c6b67640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41c00717d0 0x7f41c00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c6b67640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f41c00cca40 0x7f41c015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c6b67640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c00723c0 con 0x7f41c00717d0 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c6b67640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c0070e50 con 0x7f41c00cca40 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c6b67640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00cce70 con 0x7f41c0143c40 2023-11-24T06:28:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c5364640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41c00717d0 0x7f41c00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c5364640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41c00717d0 0x7f41c00cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33916/0 (socket says 172.21.15.64:33916) 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c5364640 1 -- 172.21.15.64:0/2494024210 learned_addr learned my addr 172.21.15.64:0/2494024210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c5b65640 1 --2- 172.21.15.64:0/2494024210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c0143c40 0x7f41c0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.791+0000 7f41c6366640 1 --2- 172.21.15.64:0/2494024210 >> v2:172.21.15.162:3300/0 conn(0x7f41c00cca40 0x7f41c015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.792+0000 7f41c5364640 1 -- 172.21.15.64:0/2494024210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c0143c40 msgr2=0x7f41c0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.792+0000 7f41c5364640 1 --2- 172.21.15.64:0/2494024210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c0143c40 0x7f41c0071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.792+0000 7f41c5364640 1 -- 172.21.15.64:0/2494024210 >> v2:172.21.15.162:3300/0 conn(0x7f41c00cca40 msgr2=0x7f41c015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.792+0000 7f41c5364640 1 --2- 172.21.15.64:0/2494024210 >> v2:172.21.15.162:3300/0 conn(0x7f41c00cca40 0x7f41c015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.792+0000 7f41c5364640 1 -- 172.21.15.64:0/2494024210 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c0144020 con 0x7f41c00717d0 2023-11-24T06:28:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.792+0000 7f41c5b65640 1 --2- 172.21.15.64:0/2494024210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c0143c40 0x7f41c0071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:51.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.793+0000 7f41c5364640 1 --2- 172.21.15.64:0/2494024210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c00717d0 0x7f41c00cc500 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41c00cc3f0 tx=0x7f41b002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6da5d0fd8d376f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.793+0000 7f41c4b63640 1 -- 172.21.15.64:0/2494024210 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b00317d0 con 0x7f41c00717d0 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c4b63640 1 -- 172.21.15.64:0/2494024210 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0031930 con 0x7f41c00717d0 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c4b63640 1 -- 172.21.15.64:0/2494024210 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0031cc0 con 0x7f41c00717d0 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c6b67640 1 -- 172.21.15.64:0/2494024210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c00717d0 msgr2=0x7f41c00cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c6b67640 1 --2- 172.21.15.64:0/2494024210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c00717d0 0x7f41c00cc500 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41c00cc3f0 tx=0x7f41b002b870 comp rx=0 tx=0).stop 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c6b67640 1 -- 172.21.15.64:0/2494024210 shutdown_connections 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c6b67640 1 --2- 172.21.15.64:0/2494024210 >> v2:172.21.15.162:3300/0 conn(0x7f41c00cca40 0x7f41c015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c6b67640 1 --2- 172.21.15.64:0/2494024210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c00717d0 0x7f41c00cc500 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c6b67640 1 --2- 172.21.15.64:0/2494024210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c0143c40 0x7f41c0071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.794+0000 7f41c6b67640 1 -- 172.21.15.64:0/2494024210 >> 172.21.15.64:0/2494024210 conn(0x7f41c006a800 msgr2=0x7f41c006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:51.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.795+0000 7f41c6b67640 1 -- 172.21.15.64:0/2494024210 shutdown_connections 2023-11-24T06:28:51.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.795+0000 7f41c6b67640 1 -- 172.21.15.64:0/2494024210 wait complete. 2023-11-24T06:28:51.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.796+0000 7f41c6b67640 1 Processor -- start 2023-11-24T06:28:51.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.797+0000 7f41c6b67640 1 -- start start 2023-11-24T06:28:51.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.797+0000 7f41c6b67640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c00717d0 0x7f41c01e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6b67640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c00cca40 0x7f41c01f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6b67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0143c40 0x7f41c01f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6b67640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c015af50 con 0x7f41c0143c40 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6b67640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00ac400 con 0x7f41c00717d0 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6b67640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00ac6c0 con 0x7f41c00cca40 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0143c40 0x7f41c01f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0143c40 0x7f41c01f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33918/0 (socket says 172.21.15.64:33918) 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 -- 172.21.15.64:0/913074364 learned_addr learned my addr 172.21.15.64:0/913074364 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c5364640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c00cca40 0x7f41c01f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c5b65640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c00717d0 0x7f41c01e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 -- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c00cca40 msgr2=0x7f41c01f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c00cca40 0x7f41c01f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 -- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c00717d0 msgr2=0x7f41c01e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c00717d0 0x7f41c01e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.798+0000 7f41c6366640 1 -- 172.21.15.64:0/913074364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00094d0 con 0x7f41c0143c40 2023-11-24T06:28:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.799+0000 7f41c5b65640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c00717d0 0x7f41c01e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.799+0000 7f41c6366640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0143c40 0x7f41c01f4b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eed0 tx=0x7f41b800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.799+0000 7f41b67fc640 1 -- 172.21.15.64:0/913074364 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8010040 con 0x7f41c0143c40 2023-11-24T06:28:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.799+0000 7f41b67fc640 1 -- 172.21.15.64:0/913074364 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b8014400 con 0x7f41c0143c40 2023-11-24T06:28:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.799+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c00ac9a0 con 0x7f41c0143c40 2023-11-24T06:28:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.799+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c00acf60 con 0x7f41c0143c40 2023-11-24T06:28:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.800+0000 7f41b67fc640 1 -- 172.21.15.64:0/913074364 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8009ea0 con 0x7f41c0143c40 2023-11-24T06:28:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.801+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c00ad1a0 con 0x7f41c0143c40 2023-11-24T06:28:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.802+0000 7f41b67fc640 1 -- 172.21.15.64:0/913074364 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41b80147b0 con 0x7f41c0143c40 2023-11-24T06:28:51.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.803+0000 7f41b67fc640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a406ae90 0x7f41a406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.803+0000 7f41b67fc640 1 -- 172.21.15.64:0/913074364 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f41b8089c60 con 0x7f41c0143c40 2023-11-24T06:28:51.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.804+0000 7f41c5b65640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a406ae90 0x7f41a406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.805+0000 7f41c5b65640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a406ae90 0x7f41a406d350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f41bc006fd0 tx=0x7f41bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:51.808+0000 7f41b67fc640 1 -- 172.21.15.64:0/913074364 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41c00ad1a0 con 0x7f41c0143c40 2023-11-24T06:28:52.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.138+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41c00ad1a0 con 0x7f41c0143c40 2023-11-24T06:28:52.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.139+0000 7f41b67fc640 1 -- 172.21.15.64:0/913074364 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f41c00ad1a0 con 0x7f41c0143c40 2023-11-24T06:28:52.142 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:52.142 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:52.142 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:28:52.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.142+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a406ae90 msgr2=0x7f41a406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.142+0000 7f41c6b67640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a406ae90 0x7f41a406d350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f41bc006fd0 tx=0x7f41bc008040 comp rx=0 tx=0).stop 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.142+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0143c40 msgr2=0x7f41c01f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.142+0000 7f41c6b67640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0143c40 0x7f41c01f4b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eed0 tx=0x7f41b800b480 comp rx=0 tx=0).stop 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 shutdown_connections 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a406ae90 0x7f41a406d350 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0143c40 0x7f41c01f4b30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41c00cca40 0x7f41c01f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 --2- 172.21.15.64:0/913074364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c00717d0 0x7f41c01e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 >> 172.21.15.64:0/913074364 conn(0x7f41c006a800 msgr2=0x7f41c00af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:52.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 shutdown_connections 2023-11-24T06:28:52.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.143+0000 7f41c6b67640 1 -- 172.21.15.64:0/913074364 wait complete. 2023-11-24T06:28:52.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.416+0000 7f158a7fc640 1 Processor -- start 2023-11-24T06:28:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.417+0000 7f158a7fc640 1 -- start start 2023-11-24T06:28:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.418+0000 7f158a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f158413df80 0x7f158415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f158a7fc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158406c470 con 0x7f158413e360 2023-11-24T06:28:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f158a7fc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158406c750 con 0x7f15840cf9a0 2023-11-24T06:28:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f158a7fc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158406ca30 con 0x7f158413df80 2023-11-24T06:28:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f1583fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f158413df80 0x7f158415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f1583fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f158413df80 0x7f158415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33924/0 (socket says 172.21.15.64:33924) 2023-11-24T06:28:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f1583fff640 1 -- 172.21.15.64:0/828272576 learned_addr learned my addr 172.21.15.64:0/828272576 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f15837fe640 1 --1- 172.21.15.64:0/828272576 >> v1:172.21.15.64:6789/0 conn(0x7f158413e360 0x7f15840cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34914/0 (socket says 172.21.15.64:34914) 2023-11-24T06:28:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f1583fff640 1 -- 172.21.15.64:0/828272576 >> v1:172.21.15.162:6789/0 conn(0x7f15840cf9a0 legacy=0x7f15840cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f1583fff640 1 -- 172.21.15.64:0/828272576 >> v1:172.21.15.64:6789/0 conn(0x7f158413e360 legacy=0x7f15840cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f1583fff640 1 -- 172.21.15.64:0/828272576 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158406cd10 con 0x7f158413df80 2023-11-24T06:28:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.419+0000 7f1582ffd640 1 -- 172.21.15.64:0/828272576 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801917470 0 0) 0x7f158406c470 con 0x7f158413e360 2023-11-24T06:28:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.421+0000 7f1583fff640 1 --2- 172.21.15.64:0/828272576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158413df80 0x7f158415bee0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1574009870 tx=0x7f157402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f733b242e943ef8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.421+0000 7f1582ffd640 1 -- 172.21.15.64:0/828272576 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157402f850 con 0x7f158413df80 2023-11-24T06:28:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.421+0000 7f1582ffd640 1 -- 172.21.15.64:0/828272576 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157402f9b0 con 0x7f158413df80 2023-11-24T06:28:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.421+0000 7f1582ffd640 1 -- 172.21.15.64:0/828272576 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157402fd40 con 0x7f158413df80 2023-11-24T06:28:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.421+0000 7f158a7fc640 1 -- 172.21.15.64:0/828272576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158413df80 msgr2=0x7f158415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.421+0000 7f158a7fc640 1 --2- 172.21.15.64:0/828272576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158413df80 0x7f158415bee0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1574009870 tx=0x7f157402ba30 comp rx=0 tx=0).stop 2023-11-24T06:28:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.422+0000 7f158a7fc640 1 -- 172.21.15.64:0/828272576 shutdown_connections 2023-11-24T06:28:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.422+0000 7f158a7fc640 1 --2- 172.21.15.64:0/828272576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158413df80 0x7f158415bee0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.422+0000 7f158a7fc640 1 -- 172.21.15.64:0/828272576 >> 172.21.15.64:0/828272576 conn(0x7f158406a800 msgr2=0x7f158406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.422+0000 7f158a7fc640 1 -- 172.21.15.64:0/828272576 shutdown_connections 2023-11-24T06:28:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.422+0000 7f158a7fc640 1 -- 172.21.15.64:0/828272576 wait complete. 2023-11-24T06:28:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.423+0000 7f158a7fc640 1 Processor -- start 2023-11-24T06:28:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.423+0000 7f158a7fc640 1 -- start start 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.424+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15840cf9a0 0x7f15841f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.424+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f158413df80 0x7f1584159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.424+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158413e360 0x7f15840aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.424+0000 7f158a7fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158413b310 con 0x7f15840cf9a0 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.424+0000 7f158a7fc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584156c70 con 0x7f158413e360 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.424+0000 7f158a7fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15841537b0 con 0x7f158413df80 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15840cf9a0 0x7f15841f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15840cf9a0 0x7f15841f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33940/0 (socket says 172.21.15.64:33940) 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 -- 172.21.15.64:0/3765042538 learned_addr learned my addr 172.21.15.64:0/3765042538 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f15837fe640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f158413df80 0x7f1584159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1588d99640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158413e360 0x7f15840aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 -- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f158413df80 msgr2=0x7f1584159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f158413df80 0x7f1584159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 -- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158413e360 msgr2=0x7f15840aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158413e360 0x7f15840aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1583fff640 1 -- 172.21.15.64:0/3765042538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740094d0 con 0x7f15840cf9a0 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.425+0000 7f1588d99640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158413e360 0x7f15840aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.426+0000 7f1583fff640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15840cf9a0 0x7f15841f0310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1574009870 tx=0x7f1574004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.426+0000 7f1580ff9640 1 -- 172.21.15.64:0/3765042538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157402f800 con 0x7f15840cf9a0 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.426+0000 7f1580ff9640 1 -- 172.21.15.64:0/3765042538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1574041400 con 0x7f15840cf9a0 2023-11-24T06:28:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.426+0000 7f1580ff9640 1 -- 172.21.15.64:0/3765042538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740417b0 con 0x7f15840cf9a0 2023-11-24T06:28:52.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.427+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1584156060 con 0x7f15840cf9a0 2023-11-24T06:28:52.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.427+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1584156620 con 0x7f15840cf9a0 2023-11-24T06:28:52.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.429+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15840675d0 con 0x7f15840cf9a0 2023-11-24T06:28:52.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.429+0000 7f1580ff9640 1 -- 172.21.15.64:0/3765042538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1574041910 con 0x7f15840cf9a0 2023-11-24T06:28:52.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.431+0000 7f1580ff9640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f157006ae40 0x7f157006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.431+0000 7f1580ff9640 1 -- 172.21.15.64:0/3765042538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f157403c070 con 0x7f15840cf9a0 2023-11-24T06:28:52.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.431+0000 7f15837fe640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f157006ae40 0x7f157006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.432+0000 7f15837fe640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f157006ae40 0x7f157006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1578007f70 tx=0x7f1578007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.438+0000 7f1580ff9640 1 -- 172.21.15.64:0/3765042538 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15740775b0 con 0x7f15840cf9a0 2023-11-24T06:28:52.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.670+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f158406df00 con 0x7f15840cf9a0 2023-11-24T06:28:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.672+0000 7f1580ff9640 1 -- 172.21.15.64:0/3765042538 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f157407abd0 con 0x7f15840cf9a0 2023-11-24T06:28:52.675 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:52.675 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.674+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f157006ae40 msgr2=0x7f157006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.674+0000 7f158a7fc640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f157006ae40 0x7f157006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1578007f70 tx=0x7f1578007f00 comp rx=0 tx=0).stop 2023-11-24T06:28:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.674+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15840cf9a0 msgr2=0x7f15841f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.674+0000 7f158a7fc640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15840cf9a0 0x7f15841f0310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1574009870 tx=0x7f1574004730 comp rx=0 tx=0).stop 2023-11-24T06:28:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 shutdown_connections 2023-11-24T06:28:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f157006ae40 0x7f157006d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158413e360 0x7f15840aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f158413df80 0x7f1584159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 --2- 172.21.15.64:0/3765042538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15840cf9a0 0x7f15841f0310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 >> 172.21.15.64:0/3765042538 conn(0x7f158406a800 msgr2=0x7f15840afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 shutdown_connections 2023-11-24T06:28:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.675+0000 7f158a7fc640 1 -- 172.21.15.64:0/3765042538 wait complete. 2023-11-24T06:28:52.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:39.545535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:40.829254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:52.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:52.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.925+0000 7f48853a5640 1 Processor -- start 2023-11-24T06:28:52.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.926+0000 7f48853a5640 1 -- start start 2023-11-24T06:28:52.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.927+0000 7f48853a5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48800cf9a0 0x7f48800cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.927+0000 7f48853a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48800b31d0 0x7f48800b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.927+0000 7f48853a5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48800d0320 con 0x7f488015d4b0 2023-11-24T06:28:52.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.927+0000 7f48853a5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800d1c60 con 0x7f48800cf9a0 2023-11-24T06:28:52.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.927+0000 7f48853a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48800b2730 con 0x7f48800b31d0 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f487ffff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f488015d4b0 0x7f48800d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:46178/0 (socket says 172.21.15.64:46178) 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f487ffff640 1 -- 172.21.15.64:0/3190849063 learned_addr learned my addr 172.21.15.64:0/3190849063 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f4877fff640 1 --2- 172.21.15.64:0/3190849063 >> v2:172.21.15.162:3300/0 conn(0x7f48800cf9a0 0x7f48800cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f4884ba4640 1 --2- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48800b31d0 0x7f48800b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f487f7fe640 1 -- 172.21.15.64:0/3190849063 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59771260 0 0) 0x7f48800d0320 con 0x7f488015d4b0 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f487f7fe640 1 -- 172.21.15.64:0/3190849063 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f486c003520 con 0x7f488015d4b0 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f4877fff640 1 -- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48800b31d0 msgr2=0x7f48800b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f4877fff640 1 --2- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48800b31d0 0x7f48800b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f4877fff640 1 -- 172.21.15.64:0/3190849063 >> v1:172.21.15.162:6789/0 conn(0x7f488015d4b0 legacy=0x7f48800d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.928+0000 7f4877fff640 1 -- 172.21.15.64:0/3190849063 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800d1ec0 con 0x7f48800cf9a0 2023-11-24T06:28:52.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.930+0000 7f4877fff640 1 --2- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800cf9a0 0x7f48800cfde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48680097a0 tx=0x7f486802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf92caf61e725ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f487f7fe640 1 -- 172.21.15.64:0/3190849063 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486802e070 con 0x7f48800cf9a0 2023-11-24T06:28:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f487f7fe640 1 -- 172.21.15.64:0/3190849063 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48680315a0 con 0x7f48800cf9a0 2023-11-24T06:28:52.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f487f7fe640 1 -- 172.21.15.64:0/3190849063 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868031930 con 0x7f48800cf9a0 2023-11-24T06:28:52.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f48853a5640 1 -- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800cf9a0 msgr2=0x7f48800cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f48853a5640 1 --2- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800cf9a0 0x7f48800cfde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48680097a0 tx=0x7f486802b840 comp rx=0 tx=0).stop 2023-11-24T06:28:52.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f48853a5640 1 -- 172.21.15.64:0/3190849063 shutdown_connections 2023-11-24T06:28:52.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f48853a5640 1 --2- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48800b31d0 0x7f48800b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.931+0000 7f48853a5640 1 --2- 172.21.15.64:0/3190849063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800cf9a0 0x7f48800cfde0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.932+0000 7f48853a5640 1 -- 172.21.15.64:0/3190849063 >> 172.21.15.64:0/3190849063 conn(0x7f48800d3000 msgr2=0x7f48800d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:52.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.932+0000 7f48853a5640 1 -- 172.21.15.64:0/3190849063 shutdown_connections 2023-11-24T06:28:52.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.932+0000 7f48853a5640 1 -- 172.21.15.64:0/3190849063 wait complete. 2023-11-24T06:28:52.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.934+0000 7f48853a5640 1 Processor -- start 2023-11-24T06:28:52.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.934+0000 7f48853a5640 1 -- start start 2023-11-24T06:28:52.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f48853a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800b31d0 0x7f488015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f48853a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48800cf9a0 0x7f488009ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f48853a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488015d4b0 0x7f488009f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f48853a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488009cce0 con 0x7f48800cf9a0 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f48853a5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488015c600 con 0x7f48800b31d0 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f48853a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48800cbb60 con 0x7f488015d4b0 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f4877fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48800cf9a0 0x7f488009ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f4884ba4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488015d4b0 0x7f488009f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f4884ba4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488015d4b0 0x7f488009f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:52142/0 (socket says 172.21.15.64:52142) 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.936+0000 7f4884ba4640 1 -- 172.21.15.64:0/1600649203 learned_addr learned my addr 172.21.15.64:0/1600649203 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f487ffff640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800b31d0 0x7f488015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f4884ba4640 1 -- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800b31d0 msgr2=0x7f488015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f4884ba4640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800b31d0 0x7f488015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f4884ba4640 1 -- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48800cf9a0 msgr2=0x7f488009ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f4884ba4640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48800cf9a0 0x7f488009ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f4884ba4640 1 -- 172.21.15.64:0/1600649203 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4868009400 con 0x7f488015d4b0 2023-11-24T06:28:52.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f4877fff640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48800cf9a0 0x7f488009ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:52.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.937+0000 7f4884ba4640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488015d4b0 0x7f488009f2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f487800efc0 tx=0x7f487800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.938+0000 7f487d7fa640 1 -- 172.21.15.64:0/1600649203 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878009ba0 con 0x7f488015d4b0 2023-11-24T06:28:52.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.938+0000 7f487d7fa640 1 -- 172.21.15.64:0/1600649203 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487800f040 con 0x7f488015d4b0 2023-11-24T06:28:52.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.938+0000 7f487d7fa640 1 -- 172.21.15.64:0/1600649203 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48780146c0 con 0x7f488015d4b0 2023-11-24T06:28:52.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.938+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488009cf30 con 0x7f488015d4b0 2023-11-24T06:28:52.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.938+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488009d4f0 con 0x7f488015d4b0 2023-11-24T06:28:52.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.939+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48800d2180 con 0x7f488015d4b0 2023-11-24T06:28:52.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.942+0000 7f487d7fa640 1 -- 172.21.15.64:0/1600649203 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4878014820 con 0x7f488015d4b0 2023-11-24T06:28:52.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.943+0000 7f487d7fa640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f486006ae40 0x7f486006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.943+0000 7f487d7fa640 1 -- 172.21.15.64:0/1600649203 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f487808a030 con 0x7f488015d4b0 2023-11-24T06:28:52.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.943+0000 7f487ffff640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f486006ae40 0x7f486006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.944+0000 7f487ffff640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f486006ae40 0x7f486006d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f488006ca80 tx=0x7f4870007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:52.953+0000 7f487d7fa640 1 -- 172.21.15.64:0/1600649203 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48800d2180 con 0x7f488015d4b0 2023-11-24T06:28:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.277+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48800d2180 con 0x7f488015d4b0 2023-11-24T06:28:53.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.278+0000 7f487d7fa640 1 -- 172.21.15.64:0/1600649203 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48800d2180 con 0x7f488015d4b0 2023-11-24T06:28:53.281 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:28:53.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.283+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f486006ae40 msgr2=0x7f486006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.283+0000 7f48853a5640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f486006ae40 0x7f486006d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f488006ca80 tx=0x7f4870007be0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.283+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488015d4b0 msgr2=0x7f488009f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.283+0000 7f48853a5640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488015d4b0 0x7f488009f2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f487800efc0 tx=0x7f487800c490 comp rx=0 tx=0).stop 2023-11-24T06:28:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.283+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 shutdown_connections 2023-11-24T06:28:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.284+0000 7f48853a5640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f486006ae40 0x7f486006d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.284+0000 7f48853a5640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488015d4b0 0x7f488009f2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.284+0000 7f48853a5640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48800cf9a0 0x7f488009ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.284+0000 7f48853a5640 1 --2- 172.21.15.64:0/1600649203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48800b31d0 0x7f488015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.284+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 >> 172.21.15.64:0/1600649203 conn(0x7f48800d3000 msgr2=0x7f48800cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.284+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 shutdown_connections 2023-11-24T06:28:53.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.284+0000 7f48853a5640 1 -- 172.21.15.64:0/1600649203 wait complete. 2023-11-24T06:28:53.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:53.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.514+0000 7ff74359e640 1 Processor -- start 2023-11-24T06:28:53.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.515+0000 7ff74359e640 1 -- start start 2023-11-24T06:28:53.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.516+0000 7ff74359e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff74415d4b0 0x7ff7440d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.516+0000 7ff74359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7440cf9a0 0x7ff7440cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.516+0000 7ff74359e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440d0320 con 0x7ff7440cddf0 2023-11-24T06:28:53.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.516+0000 7ff74359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440b2730 con 0x7ff7440cf9a0 2023-11-24T06:28:53.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.516+0000 7ff74359e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440d1ec0 con 0x7ff74415d4b0 2023-11-24T06:28:53.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.516+0000 7ff741d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7440cf9a0 0x7ff7440cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff741d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7440cf9a0 0x7ff7440cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50604/0 (socket says 172.21.15.64:50604) 2023-11-24T06:28:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff741d9b640 1 -- 172.21.15.64:0/3695589859 learned_addr learned my addr 172.21.15.64:0/3695589859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff741d9b640 1 -- 172.21.15.64:0/3695589859 >> v2:172.21.15.162:3300/0 conn(0x7ff74415d4b0 msgr2=0x7ff7440d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff74259c640 1 --2- 172.21.15.64:0/3695589859 >> v2:172.21.15.162:3300/0 conn(0x7ff74415d4b0 0x7ff7440d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff741d9b640 1 --2- 172.21.15.64:0/3695589859 >> v2:172.21.15.162:3300/0 conn(0x7ff74415d4b0 0x7ff7440d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff741d9b640 1 -- 172.21.15.64:0/3695589859 >> v1:172.21.15.162:6789/0 conn(0x7ff7440cddf0 legacy=0x7ff7440ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff741d9b640 1 -- 172.21.15.64:0/3695589859 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440b29d0 con 0x7ff7440cf9a0 2023-11-24T06:28:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.517+0000 7ff74259c640 1 --2- 172.21.15.64:0/3695589859 >> v2:172.21.15.162:3300/0 conn(0x7ff74415d4b0 0x7ff7440d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.519+0000 7ff741d9b640 1 --2- 172.21.15.64:0/3695589859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff7440cfde0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff74408b5b0 tx=0x7ff73802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7184729196eb4f3e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.519+0000 7ff74159a640 1 -- 172.21.15.64:0/3695589859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738031870 con 0x7ff7440cf9a0 2023-11-24T06:28:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74159a640 1 -- 172.21.15.64:0/3695589859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7380319d0 con 0x7ff7440cf9a0 2023-11-24T06:28:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74159a640 1 -- 172.21.15.64:0/3695589859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738031d60 con 0x7ff7440cf9a0 2023-11-24T06:28:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 -- 172.21.15.64:0/3695589859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 msgr2=0x7ff7440cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 --2- 172.21.15.64:0/3695589859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff7440cfde0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff74408b5b0 tx=0x7ff73802b910 comp rx=0 tx=0).stop 2023-11-24T06:28:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 -- 172.21.15.64:0/3695589859 shutdown_connections 2023-11-24T06:28:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 --2- 172.21.15.64:0/3695589859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff7440cfde0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 --2- 172.21.15.64:0/3695589859 >> v2:172.21.15.162:3300/0 conn(0x7ff74415d4b0 0x7ff7440d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 -- 172.21.15.64:0/3695589859 >> 172.21.15.64:0/3695589859 conn(0x7ff7440d3000 msgr2=0x7ff7440d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 -- 172.21.15.64:0/3695589859 shutdown_connections 2023-11-24T06:28:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.520+0000 7ff74359e640 1 -- 172.21.15.64:0/3695589859 wait complete. 2023-11-24T06:28:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.523+0000 7ff74359e640 1 Processor -- start 2023-11-24T06:28:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.523+0000 7ff74359e640 1 -- start start 2023-11-24T06:28:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff74359e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7440cddf0 0x7ff74414d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff74359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff74409f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff74359e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff74415d4b0 0x7ff74409fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff74359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440a0180 con 0x7ff7440cf9a0 2023-11-24T06:28:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff74359e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440a0460 con 0x7ff7440cddf0 2023-11-24T06:28:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff74359e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440a0740 con 0x7ff74415d4b0 2023-11-24T06:28:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff741d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff74409f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff741d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff74409f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50610/0 (socket says 172.21.15.64:50610) 2023-11-24T06:28:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.524+0000 7ff741d9b640 1 -- 172.21.15.64:0/2554552628 learned_addr learned my addr 172.21.15.64:0/2554552628 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff74259c640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7440cddf0 0x7ff74414d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff742d9d640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff74415d4b0 0x7ff74409fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff741d9b640 1 -- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff74415d4b0 msgr2=0x7ff74409fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff741d9b640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff74415d4b0 0x7ff74409fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff741d9b640 1 -- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7440cddf0 msgr2=0x7ff74414d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff741d9b640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7440cddf0 0x7ff74414d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff741d9b640 1 -- 172.21.15.64:0/2554552628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380094d0 con 0x7ff7440cf9a0 2023-11-24T06:28:53.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.525+0000 7ff741d9b640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff74409f5c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff738000c00 tx=0x7ff7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:53.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.526+0000 7ff732ffd640 1 -- 172.21.15.64:0/2554552628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73803adf0 con 0x7ff7440cf9a0 2023-11-24T06:28:53.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.526+0000 7ff732ffd640 1 -- 172.21.15.64:0/2554552628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7380443f0 con 0x7ff7440cf9a0 2023-11-24T06:28:53.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.526+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74415da10 con 0x7ff7440cf9a0 2023-11-24T06:28:53.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.526+0000 7ff732ffd640 1 -- 172.21.15.64:0/2554552628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7380447a0 con 0x7ff7440cf9a0 2023-11-24T06:28:53.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.526+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74414d910 con 0x7ff7440cf9a0 2023-11-24T06:28:53.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.527+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff744067450 con 0x7ff7440cf9a0 2023-11-24T06:28:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.529+0000 7ff732ffd640 1 -- 172.21.15.64:0/2554552628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff73804d450 con 0x7ff7440cf9a0 2023-11-24T06:28:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.530+0000 7ff732ffd640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff72406ae90 0x7ff72406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.530+0000 7ff732ffd640 1 -- 172.21.15.64:0/2554552628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7ff73802d040 con 0x7ff7440cf9a0 2023-11-24T06:28:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.530+0000 7ff74259c640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff72406ae90 0x7ff72406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.531+0000 7ff74259c640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff72406ae90 0x7ff72406d350 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff73400aac0 tx=0x7ff734008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.534+0000 7ff732ffd640 1 -- 172.21.15.64:0/2554552628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff738076c30 con 0x7ff7440cf9a0 2023-11-24T06:28:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.752+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7440b4110 con 0x7ff7440cf9a0 2023-11-24T06:28:53.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.753+0000 7ff732ffd640 1 -- 172.21.15.64:0/2554552628 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff73807a250 con 0x7ff7440cf9a0 2023-11-24T06:28:53.756 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:28:53.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.755+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff72406ae90 msgr2=0x7ff72406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.755+0000 7ff74359e640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff72406ae90 0x7ff72406d350 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff73400aac0 tx=0x7ff734008040 comp rx=0 tx=0).stop 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.755+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 msgr2=0x7ff74409f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.755+0000 7ff74359e640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff74409f5c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff738000c00 tx=0x7ff7380046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 shutdown_connections 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff72406ae90 0x7ff72406d350 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff74415d4b0 0x7ff74409fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7440cf9a0 0x7ff74409f5c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 --2- 172.21.15.64:0/2554552628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7440cddf0 0x7ff74414d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 >> 172.21.15.64:0/2554552628 conn(0x7ff7440d3000 msgr2=0x7ff7440b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:53.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 shutdown_connections 2023-11-24T06:28:53.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:53.756+0000 7ff74359e640 1 -- 172.21.15.64:0/2554552628 wait complete. 2023-11-24T06:28:53.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:54.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.015+0000 7f5fa7d43640 1 Processor -- start 2023-11-24T06:28:54.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.015+0000 7f5fa7d43640 1 -- start start 2023-11-24T06:28:54.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.017+0000 7f5fa7d43640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5fa015e3b0 0x7f5fa0164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.017+0000 7f5fa7d43640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa00b1ee0 con 0x7f5fa0143ae0 2023-11-24T06:28:54.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.017+0000 7f5fa7d43640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa00b2780 con 0x7f5fa00cf9a0 2023-11-24T06:28:54.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.017+0000 7f5fa7d43640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa015e990 con 0x7f5fa015e3b0 2023-11-24T06:28:54.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.018+0000 7f5fa52de640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5fa00cf9a0 0x7f5fa015dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:46188/0 (socket says 172.21.15.64:46188) 2023-11-24T06:28:54.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.018+0000 7f5fa52de640 1 -- 172.21.15.64:0/2517379184 learned_addr learned my addr 172.21.15.64:0/2517379184 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:54.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.018+0000 7f5fa62e0640 1 --2- 172.21.15.64:0/2517379184 >> v2:172.21.15.64:3300/0 conn(0x7f5fa015e3b0 0x7f5fa0164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.018+0000 7f5fa62e0640 1 -- 172.21.15.64:0/2517379184 >> v1:172.21.15.162:6789/0 conn(0x7f5fa00cf9a0 legacy=0x7f5fa015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.018+0000 7f5fa62e0640 1 -- 172.21.15.64:0/2517379184 >> v1:172.21.15.64:6789/0 conn(0x7f5fa0143ae0 legacy=0x7f5fa00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.019+0000 7f5fa62e0640 1 -- 172.21.15.64:0/2517379184 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa00b2130 con 0x7f5fa015e3b0 2023-11-24T06:28:54.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.020+0000 7f5fa62e0640 1 --2- 172.21.15.64:0/2517379184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa0164de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009d90 tx=0x7f5f9802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534f980ecfd2fd64 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.020+0000 7f5fa4add640 1 -- 172.21.15.64:0/2517379184 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9802ccf0 con 0x7f5fa015e3b0 2023-11-24T06:28:54.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.020+0000 7f5fa4add640 1 -- 172.21.15.64:0/2517379184 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9802ce50 con 0x7f5fa015e3b0 2023-11-24T06:28:54.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.020+0000 7f5fa4add640 1 -- 172.21.15.64:0/2517379184 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98034660 con 0x7f5fa015e3b0 2023-11-24T06:28:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.020+0000 7f5fa7d43640 1 -- 172.21.15.64:0/2517379184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 msgr2=0x7f5fa0164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.020+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/2517379184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa0164de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009d90 tx=0x7f5f9802bf40 comp rx=0 tx=0).stop 2023-11-24T06:28:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.021+0000 7f5fa7d43640 1 -- 172.21.15.64:0/2517379184 shutdown_connections 2023-11-24T06:28:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.021+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/2517379184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa0164de0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.021+0000 7f5fa7d43640 1 -- 172.21.15.64:0/2517379184 >> 172.21.15.64:0/2517379184 conn(0x7f5fa006a800 msgr2=0x7f5fa006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.021+0000 7f5fa7d43640 1 -- 172.21.15.64:0/2517379184 shutdown_connections 2023-11-24T06:28:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.021+0000 7f5fa7d43640 1 -- 172.21.15.64:0/2517379184 wait complete. 2023-11-24T06:28:54.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.023+0000 7f5fa7d43640 1 Processor -- start 2023-11-24T06:28:54.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.024+0000 7f5fa7d43640 1 -- start start 2023-11-24T06:28:54.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa7d43640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa00cf9a0 0x7f5fa01f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa7d43640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa0143ae0 0x7f5fa00c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa7d43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa00ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa7d43640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa00c5270 con 0x7f5fa015e3b0 2023-11-24T06:28:54.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa7d43640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa00ac480 con 0x7f5fa0143ae0 2023-11-24T06:28:54.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa7d43640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa00ac760 con 0x7f5fa00cf9a0 2023-11-24T06:28:54.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa62e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa00ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa62e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa00ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50624/0 (socket says 172.21.15.64:50624) 2023-11-24T06:28:54.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.026+0000 7f5fa62e0640 1 -- 172.21.15.64:0/1675636639 learned_addr learned my addr 172.21.15.64:0/1675636639 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:54.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa52de640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa0143ae0 0x7f5fa00c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa5adf640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa00cf9a0 0x7f5fa01f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa62e0640 1 -- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa00cf9a0 msgr2=0x7f5fa01f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa62e0640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa00cf9a0 0x7f5fa01f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa62e0640 1 -- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa0143ae0 msgr2=0x7f5fa00c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa62e0640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa0143ae0 0x7f5fa00c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa62e0640 1 -- 172.21.15.64:0/1675636639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f98009a70 con 0x7f5fa015e3b0 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa52de640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa0143ae0 0x7f5fa00c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.027+0000 7f5fa62e0640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa00ad520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009d90 tx=0x7f5f98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.028+0000 7f5f967fc640 1 -- 172.21.15.64:0/1675636639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9803e610 con 0x7f5fa015e3b0 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.028+0000 7f5f967fc640 1 -- 172.21.15.64:0/1675636639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9803e770 con 0x7f5fa015e3b0 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.028+0000 7f5f967fc640 1 -- 172.21.15.64:0/1675636639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9803eb20 con 0x7f5fa015e3b0 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.028+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa00aca40 con 0x7f5fa015e3b0 2023-11-24T06:28:54.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.028+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa00ac0a0 con 0x7f5fa015e3b0 2023-11-24T06:28:54.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.030+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa00ad240 con 0x7f5fa015e3b0 2023-11-24T06:28:54.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.032+0000 7f5f967fc640 1 -- 172.21.15.64:0/1675636639 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f9803b050 con 0x7f5fa015e3b0 2023-11-24T06:28:54.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.033+0000 7f5f967fc640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f8406ae40 0x7f5f8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.033+0000 7f5f967fc640 1 -- 172.21.15.64:0/1675636639 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f5f980ac090 con 0x7f5fa015e3b0 2023-11-24T06:28:54.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.034+0000 7f5fa5adf640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f8406ae40 0x7f5f8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.035+0000 7f5fa5adf640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f8406ae40 0x7f5f8406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001640 tx=0x7f5f9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.038+0000 7f5f967fc640 1 -- 172.21.15.64:0/1675636639 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5fa00ad240 con 0x7f5fa015e3b0 2023-11-24T06:28:54.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.285+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5fa00ad240 con 0x7f5fa015e3b0 2023-11-24T06:28:54.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.285+0000 7f5f967fc640 1 -- 172.21.15.64:0/1675636639 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f98039020 con 0x7f5fa015e3b0 2023-11-24T06:28:54.289 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:28:54.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.288+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f8406ae40 msgr2=0x7f5f8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.288+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f8406ae40 0x7f5f8406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001640 tx=0x7f5f9c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:54.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.288+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 msgr2=0x7f5fa00ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.288+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa00ad520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009d90 tx=0x7f5f98004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:54.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.289+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 shutdown_connections 2023-11-24T06:28:54.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.289+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5f8406ae40 0x7f5f8406d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.289+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa015e3b0 0x7f5fa00ad520 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.289+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa0143ae0 0x7f5fa00c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.289+0000 7f5fa7d43640 1 --2- 172.21.15.64:0/1675636639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa00cf9a0 0x7f5fa01f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.289+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 >> 172.21.15.64:0/1675636639 conn(0x7f5fa006a800 msgr2=0x7f5fa015b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:54.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.289+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 shutdown_connections 2023-11-24T06:28:54.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.290+0000 7f5fa7d43640 1 -- 172.21.15.64:0/1675636639 wait complete. 2023-11-24T06:28:54.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:54.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.559+0000 7f7053b41640 1 Processor -- start 2023-11-24T06:28:54.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.559+0000 7f7053b41640 1 -- start start 2023-11-24T06:28:54.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.562+0000 7f7053b41640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.562+0000 7f7053b41640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c071de0 con 0x7f704c0aede0 2023-11-24T06:28:54.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.562+0000 7f7053b41640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c0af270 con 0x7f704c0d0b60 2023-11-24T06:28:54.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.562+0000 7f7053b41640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c072030 con 0x7f704c06a3e0 2023-11-24T06:28:54.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.562+0000 7f70520de640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f704c0aede0 0x7f704c06f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39052/0 (socket says 172.21.15.64:39052) 2023-11-24T06:28:54.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.562+0000 7f70520de640 1 -- 172.21.15.64:0/344018637 learned_addr learned my addr 172.21.15.64:0/344018637 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:54.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.562+0000 7f70518dd640 1 --2- 172.21.15.64:0/344018637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.563+0000 7f70518dd640 1 -- 172.21.15.64:0/344018637 >> v1:172.21.15.162:6789/0 conn(0x7f704c0d0b60 legacy=0x7f704c06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.563+0000 7f70518dd640 1 -- 172.21.15.64:0/344018637 >> v1:172.21.15.64:6789/0 conn(0x7f704c0aede0 legacy=0x7f704c06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.563+0000 7f70518dd640 1 -- 172.21.15.64:0/344018637 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c072310 con 0x7f704c06a3e0 2023-11-24T06:28:54.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.563+0000 7f70508db640 1 -- 172.21.15.64:0/344018637 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183996138 0 0) 0x7f704c071de0 con 0x7f704c0aede0 2023-11-24T06:28:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.564+0000 7f70518dd640 1 --2- 172.21.15.64:0/344018637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c0ac8d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f70480097d0 tx=0x7f704802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee03a148a899856 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f70508db640 1 -- 172.21.15.64:0/344018637 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048031890 con 0x7f704c06a3e0 2023-11-24T06:28:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f70508db640 1 -- 172.21.15.64:0/344018637 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70480319f0 con 0x7f704c06a3e0 2023-11-24T06:28:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f70508db640 1 -- 172.21.15.64:0/344018637 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048031d80 con 0x7f704c06a3e0 2023-11-24T06:28:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f7053b41640 1 -- 172.21.15.64:0/344018637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 msgr2=0x7f704c0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f7053b41640 1 --2- 172.21.15.64:0/344018637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c0ac8d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f70480097d0 tx=0x7f704802b930 comp rx=0 tx=0).stop 2023-11-24T06:28:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f7053b41640 1 -- 172.21.15.64:0/344018637 shutdown_connections 2023-11-24T06:28:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f7053b41640 1 --2- 172.21.15.64:0/344018637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c0ac8d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.565+0000 7f7053b41640 1 -- 172.21.15.64:0/344018637 >> 172.21.15.64:0/344018637 conn(0x7f704c0cfef0 msgr2=0x7f704c0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.566+0000 7f7053b41640 1 -- 172.21.15.64:0/344018637 shutdown_connections 2023-11-24T06:28:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.566+0000 7f7053b41640 1 -- 172.21.15.64:0/344018637 wait complete. 2023-11-24T06:28:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.568+0000 7f7053b41640 1 Processor -- start 2023-11-24T06:28:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.568+0000 7f7053b41640 1 -- start start 2023-11-24T06:28:54.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f7053b41640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f7053b41640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0aede0 0x7f704c145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f7053b41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f704c0d0b60 0x7f704c15b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f7053b41640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c14c5f0 con 0x7f704c0d0b60 2023-11-24T06:28:54.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f7053b41640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c149d70 con 0x7f704c0aede0 2023-11-24T06:28:54.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f7053b41640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c14a050 con 0x7f704c06a3e0 2023-11-24T06:28:54.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f70510dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0aede0 0x7f704c145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f70510dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0aede0 0x7f704c145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:43864/0 (socket says 172.21.15.64:43864) 2023-11-24T06:28:54.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.569+0000 7f70510dc640 1 -- 172.21.15.64:0/539089876 learned_addr learned my addr 172.21.15.64:0/539089876 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:54.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.570+0000 7f70518dd640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.570+0000 7f70520de640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f704c0d0b60 0x7f704c15b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.570+0000 7f70518dd640 1 -- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0aede0 msgr2=0x7f704c145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.570+0000 7f70518dd640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0aede0 0x7f704c145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.570+0000 7f70518dd640 1 -- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f704c0d0b60 msgr2=0x7f704c15b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.570+0000 7f70518dd640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f704c0d0b60 0x7f704c15b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.570+0000 7f70518dd640 1 -- 172.21.15.64:0/539089876 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7048009400 con 0x7f704c06a3e0 2023-11-24T06:28:54.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.571+0000 7f70518dd640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c15c9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7048033000 tx=0x7f70480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.571+0000 7f70427fc640 1 -- 172.21.15.64:0/539089876 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048031830 con 0x7f704c06a3e0 2023-11-24T06:28:54.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.571+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c14a330 con 0x7f704c06a3e0 2023-11-24T06:28:54.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.572+0000 7f70427fc640 1 -- 172.21.15.64:0/539089876 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7048004030 con 0x7f704c06a3e0 2023-11-24T06:28:54.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.572+0000 7f70427fc640 1 -- 172.21.15.64:0/539089876 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70480043e0 con 0x7f704c06a3e0 2023-11-24T06:28:54.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.572+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c14a8f0 con 0x7f704c06a3e0 2023-11-24T06:28:54.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.573+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c0675d0 con 0x7f704c06a3e0 2023-11-24T06:28:54.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.575+0000 7f70427fc640 1 -- 172.21.15.64:0/539089876 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7048003610 con 0x7f704c06a3e0 2023-11-24T06:28:54.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.576+0000 7f70427fc640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f703006ae40 0x7f703006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.576+0000 7f70427fc640 1 -- 172.21.15.64:0/539089876 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f70480aca30 con 0x7f704c06a3e0 2023-11-24T06:28:54.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.577+0000 7f70510dc640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f703006ae40 0x7f703006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.578+0000 7f70510dc640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f703006ae40 0x7f703006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f704c0ada80 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.584+0000 7f70427fc640 1 -- 172.21.15.64:0/539089876 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7048077b10 con 0x7f704c06a3e0 2023-11-24T06:28:54.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.928+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f704c0b0610 con 0x7f704c06a3e0 2023-11-24T06:28:54.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.930+0000 7f70427fc640 1 -- 172.21.15.64:0/539089876 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f7048035090 con 0x7f704c06a3e0 2023-11-24T06:28:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:54.933 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:54.933 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:28:54.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.932+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f703006ae40 msgr2=0x7f703006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.932+0000 7f7053b41640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f703006ae40 0x7f703006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f704c0ada80 tx=0x7f703c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:54.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.932+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 msgr2=0x7f704c15c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.932+0000 7f7053b41640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c15c9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7048033000 tx=0x7f70480046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 shutdown_connections 2023-11-24T06:28:54.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f703006ae40 0x7f703006d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f704c0d0b60 0x7f704c15b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0aede0 0x7f704c145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 --2- 172.21.15.64:0/539089876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c06a3e0 0x7f704c15c9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 >> 172.21.15.64:0/539089876 conn(0x7f704c0cfef0 msgr2=0x7f704c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:54.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 shutdown_connections 2023-11-24T06:28:54.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:54.933+0000 7f7053b41640 1 -- 172.21.15.64:0/539089876 wait complete. 2023-11-24T06:28:54.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.179+0000 7f1942c1a640 1 Processor -- start 2023-11-24T06:28:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.179+0000 7f1942c1a640 1 -- start start 2023-11-24T06:28:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.181+0000 7f1942c1a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.181+0000 7f1942c1a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c0702e0 con 0x7f193c162810 2023-11-24T06:28:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.181+0000 7f1942c1a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c0705c0 con 0x7f193c0b3650 2023-11-24T06:28:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.181+0000 7f1942c1a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c0708a0 con 0x7f193c162430 2023-11-24T06:28:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.181+0000 7f1941417640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f193c162810 0x7f193c071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39062/0 (socket says 172.21.15.64:39062) 2023-11-24T06:28:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.181+0000 7f1941417640 1 -- 172.21.15.64:0/1536794667 learned_addr learned my addr 172.21.15.64:0/1536794667 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.181+0000 7f1941c18640 1 --2- 172.21.15.64:0/1536794667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.182+0000 7f1940c16640 1 -- 172.21.15.64:0/1536794667 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792578810 0 0) 0x7f193c0702e0 con 0x7f193c162810 2023-11-24T06:28:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.182+0000 7f1940c16640 1 -- 172.21.15.64:0/1536794667 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1928003520 con 0x7f193c162810 2023-11-24T06:28:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.182+0000 7f1940c16640 1 -- 172.21.15.64:0/1536794667 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229148955 0 0) 0x7f193c0705c0 con 0x7f193c0b3650 2023-11-24T06:28:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.182+0000 7f1940c16640 1 -- 172.21.15.64:0/1536794667 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f193c0702e0 con 0x7f193c0b3650 2023-11-24T06:28:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.182+0000 7f1941c18640 1 -- 172.21.15.64:0/1536794667 >> v1:172.21.15.162:6789/0 conn(0x7f193c0b3650 legacy=0x7f193c06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.182+0000 7f1941c18640 1 -- 172.21.15.64:0/1536794667 >> v1:172.21.15.64:6789/0 conn(0x7f193c162810 legacy=0x7f193c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.182+0000 7f1941c18640 1 -- 172.21.15.64:0/1536794667 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c070b80 con 0x7f193c162430 2023-11-24T06:28:55.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.183+0000 7f1941c18640 1 --2- 172.21.15.64:0/1536794667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b3110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19380097d0 tx=0x7f193802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af4bd1e6786169 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.184+0000 7f1940c16640 1 -- 172.21.15.64:0/1536794667 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938031890 con 0x7f193c162430 2023-11-24T06:28:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.184+0000 7f1940c16640 1 -- 172.21.15.64:0/1536794667 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19380319f0 con 0x7f193c162430 2023-11-24T06:28:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.184+0000 7f1940c16640 1 -- 172.21.15.64:0/1536794667 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938031d60 con 0x7f193c162430 2023-11-24T06:28:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.184+0000 7f1942c1a640 1 -- 172.21.15.64:0/1536794667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 msgr2=0x7f193c0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.184+0000 7f1942c1a640 1 --2- 172.21.15.64:0/1536794667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b3110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19380097d0 tx=0x7f193802b930 comp rx=0 tx=0).stop 2023-11-24T06:28:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.185+0000 7f1942c1a640 1 -- 172.21.15.64:0/1536794667 shutdown_connections 2023-11-24T06:28:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.185+0000 7f1942c1a640 1 --2- 172.21.15.64:0/1536794667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b3110 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.185+0000 7f1942c1a640 1 -- 172.21.15.64:0/1536794667 >> 172.21.15.64:0/1536794667 conn(0x7f193c06a800 msgr2=0x7f193c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.185+0000 7f1942c1a640 1 -- 172.21.15.64:0/1536794667 shutdown_connections 2023-11-24T06:28:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.185+0000 7f1942c1a640 1 -- 172.21.15.64:0/1536794667 wait complete. 2023-11-24T06:28:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.188+0000 7f1942c1a640 1 Processor -- start 2023-11-24T06:28:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.188+0000 7f1942c1a640 1 -- start start 2023-11-24T06:28:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1942c1a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f193c0b3650 0x7f193c0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1942c1a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1942c1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f193c1ebf30 0x7f193c1ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1942c1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c15a9c0 con 0x7f193c1ebf30 2023-11-24T06:28:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1942c1a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c143ac0 con 0x7f193c0b3650 2023-11-24T06:28:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1942c1a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c15a370 con 0x7f193c162430 2023-11-24T06:28:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1941c18640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f193c0b3650 0x7f193c0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1942419640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f193c1ebf30 0x7f193c1ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1941c18640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f193c0b3650 0x7f193c0b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:43866/0 (socket says 172.21.15.64:43866) 2023-11-24T06:28:55.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1941c18640 1 -- 172.21.15.64:0/2613286907 learned_addr learned my addr 172.21.15.64:0/2613286907 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:55.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.189+0000 7f1941417640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.190+0000 7f1941c18640 1 -- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 msgr2=0x7f193c0b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.190+0000 7f1941c18640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.190+0000 7f1941c18640 1 -- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f193c1ebf30 msgr2=0x7f193c1ec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.190+0000 7f1941c18640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f193c1ebf30 0x7f193c1ec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.190+0000 7f1941c18640 1 -- 172.21.15.64:0/2613286907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1938009400 con 0x7f193c0b3650 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.190+0000 7f1942419640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f193c1ebf30 0x7f193c1ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.190+0000 7f1941c18640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f193c0b3650 0x7f193c0b0a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1938000c00 tx=0x7f19380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.191+0000 7f19327fc640 1 -- 172.21.15.64:0/2613286907 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193803bbf0 con 0x7f193c0b3650 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.191+0000 7f19327fc640 1 -- 172.21.15.64:0/2613286907 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193803bd50 con 0x7f193c0b3650 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.191+0000 7f19327fc640 1 -- 172.21.15.64:0/2613286907 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938045630 con 0x7f193c0b3650 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.191+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c15a650 con 0x7f193c0b3650 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.191+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c140c90 con 0x7f193c0b3650 2023-11-24T06:28:55.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.192+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c067450 con 0x7f193c0b3650 2023-11-24T06:28:55.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.194+0000 7f19327fc640 1 -- 172.21.15.64:0/2613286907 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1938004030 con 0x7f193c0b3650 2023-11-24T06:28:55.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.195+0000 7f19327fc640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192006af40 0x7f192006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.195+0000 7f19327fc640 1 -- 172.21.15.64:0/2613286907 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f19380abe90 con 0x7f193c0b3650 2023-11-24T06:28:55.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.195+0000 7f1941417640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192006af40 0x7f192006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.196+0000 7f1941417640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192006af40 0x7f192006d400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f193c165f30 tx=0x7f192c007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.201+0000 7f19327fc640 1 -- 172.21.15.64:0/2613286907 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1938076f70 con 0x7f193c0b3650 2023-11-24T06:28:55.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.490+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f193c165f60 con 0x7f193c0b3650 2023-11-24T06:28:55.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.493+0000 7f19327fc640 1 -- 172.21.15.64:0/2613286907 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f193807a590 con 0x7f193c0b3650 2023-11-24T06:28:55.496 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:55.496 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:55.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.495+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192006af40 msgr2=0x7f192006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.495+0000 7f1942c1a640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192006af40 0x7f192006d400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f193c165f30 tx=0x7f192c007d30 comp rx=0 tx=0).stop 2023-11-24T06:28:55.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.495+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f193c0b3650 msgr2=0x7f193c0b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.495+0000 7f1942c1a640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f193c0b3650 0x7f193c0b0a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1938000c00 tx=0x7f19380046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.495+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 shutdown_connections 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.495+0000 7f1942c1a640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f192006af40 0x7f192006d400 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.496+0000 7f1942c1a640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f193c1ebf30 0x7f193c1ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.496+0000 7f1942c1a640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f193c162430 0x7f193c0b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.496+0000 7f1942c1a640 1 --2- 172.21.15.64:0/2613286907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f193c0b3650 0x7f193c0b0a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.496+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 >> 172.21.15.64:0/2613286907 conn(0x7f193c06a800 msgr2=0x7f193c0b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.496+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 shutdown_connections 2023-11-24T06:28:55.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.496+0000 7f1942c1a640 1 -- 172.21.15.64:0/2613286907 wait complete. 2023-11-24T06:28:55.516 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:28:55.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:55.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.752+0000 7f540b57a640 1 Processor -- start 2023-11-24T06:28:55.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.752+0000 7f540b57a640 1 -- start start 2023-11-24T06:28:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f540b57a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54040d0100 0x7f54040b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f540b57a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0f40 0x7f54040d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f540b57a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54040d0a10 con 0x7f5404160920 2023-11-24T06:28:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f540b57a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54040cf080 con 0x7f54040d0100 2023-11-24T06:28:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f540b57a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54040c6be0 con 0x7f54040d0f40 2023-11-24T06:28:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f5409d77640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54040d0100 0x7f54040b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f5409d77640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54040d0100 0x7f54040b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50640/0 (socket says 172.21.15.64:50640) 2023-11-24T06:28:55.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.754+0000 7f5409d77640 1 -- 172.21.15.64:0/1620968339 learned_addr learned my addr 172.21.15.64:0/1620968339 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:55.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.755+0000 7f540ad79640 1 --2- 172.21.15.64:0/1620968339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0f40 0x7f54040d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.755+0000 7f5409d77640 1 -- 172.21.15.64:0/1620968339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0f40 msgr2=0x7f54040d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.755+0000 7f5409d77640 1 --2- 172.21.15.64:0/1620968339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0f40 0x7f54040d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.755+0000 7f5409d77640 1 -- 172.21.15.64:0/1620968339 >> v1:172.21.15.162:6789/0 conn(0x7f5404160920 legacy=0x7f54040cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.755+0000 7f5409d77640 1 -- 172.21.15.64:0/1620968339 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040ce7e0 con 0x7f54040d0100 2023-11-24T06:28:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.755+0000 7f540ad79640 1 --2- 172.21.15.64:0/1620968339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0f40 0x7f54040d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.756+0000 7f5409d77640 1 --2- 172.21.15.64:0/1620968339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0100 0x7f54040b3f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009870 tx=0x7f53f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a977abf7b99a1710 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.756+0000 7f5409576640 1 -- 172.21.15.64:0/1620968339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4031640 con 0x7f54040d0100 2023-11-24T06:28:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.757+0000 7f5409576640 1 -- 172.21.15.64:0/1620968339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f40317a0 con 0x7f54040d0100 2023-11-24T06:28:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.757+0000 7f5409576640 1 -- 172.21.15.64:0/1620968339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4031b30 con 0x7f54040d0100 2023-11-24T06:28:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 -- 172.21.15.64:0/1620968339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0100 msgr2=0x7f54040b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 --2- 172.21.15.64:0/1620968339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0100 0x7f54040b3f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009870 tx=0x7f53f402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 -- 172.21.15.64:0/1620968339 shutdown_connections 2023-11-24T06:28:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 --2- 172.21.15.64:0/1620968339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0f40 0x7f54040d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 --2- 172.21.15.64:0/1620968339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0100 0x7f54040b3f30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 -- 172.21.15.64:0/1620968339 >> 172.21.15.64:0/1620968339 conn(0x7f54040d3090 msgr2=0x7f54040d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 -- 172.21.15.64:0/1620968339 shutdown_connections 2023-11-24T06:28:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.758+0000 7f540b57a640 1 -- 172.21.15.64:0/1620968339 wait complete. 2023-11-24T06:28:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.760+0000 7f540b57a640 1 Processor -- start 2023-11-24T06:28:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.760+0000 7f540b57a640 1 -- start start 2023-11-24T06:28:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f540b57a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0100 0x7f540414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f540b57a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0f40 0x7f54040ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f540b57a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5404160920 0x7f540415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f540b57a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54040cb840 con 0x7f54040d0f40 2023-11-24T06:28:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f540b57a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041581c0 con 0x7f5404160920 2023-11-24T06:28:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f540b57a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5404161a00 con 0x7f54040d0100 2023-11-24T06:28:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f5409d77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0f40 0x7f54040ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f5409d77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0f40 0x7f54040ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50644/0 (socket says 172.21.15.64:50644) 2023-11-24T06:28:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.761+0000 7f5409d77640 1 -- 172.21.15.64:0/3846630444 learned_addr learned my addr 172.21.15.64:0/3846630444 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f540a578640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0100 0x7f540414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f540ad79640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5404160920 0x7f540415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f5409d77640 1 -- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0100 msgr2=0x7f540414d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f5409d77640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0100 0x7f540414d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f5409d77640 1 -- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5404160920 msgr2=0x7f540415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f5409d77640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5404160920 0x7f540415ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f5409d77640 1 -- 172.21.15.64:0/3846630444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40094d0 con 0x7f54040d0f40 2023-11-24T06:28:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.762+0000 7f5409d77640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0f40 0x7f54040ccd20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009870 tx=0x7f53f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.763+0000 7f53faffd640 1 -- 172.21.15.64:0/3846630444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f403ac60 con 0x7f54040d0f40 2023-11-24T06:28:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.763+0000 7f53faffd640 1 -- 172.21.15.64:0/3846630444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f403ade0 con 0x7f54040d0f40 2023-11-24T06:28:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.763+0000 7f53faffd640 1 -- 172.21.15.64:0/3846630444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4044630 con 0x7f54040d0f40 2023-11-24T06:28:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.763+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5404161ce0 con 0x7f54040d0f40 2023-11-24T06:28:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.763+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5404158740 con 0x7f54040d0f40 2023-11-24T06:28:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.764+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5404158930 con 0x7f54040d0f40 2023-11-24T06:28:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.766+0000 7f53faffd640 1 -- 172.21.15.64:0/3846630444 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53f402f050 con 0x7f54040d0f40 2023-11-24T06:28:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.767+0000 7f53faffd640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f53e806ae40 0x7f53e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.767+0000 7f53faffd640 1 -- 172.21.15.64:0/3846630444 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f53f40ab770 con 0x7f54040d0f40 2023-11-24T06:28:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.767+0000 7f540a578640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f53e806ae40 0x7f53e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.768+0000 7f540a578640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f53e806ae40 0x7f53e806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5400006fd0 tx=0x7f5400008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:55.775+0000 7f53faffd640 1 -- 172.21.15.64:0/3846630444 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53f4076830 con 0x7f54040d0f40 2023-11-24T06:28:56.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:56.048 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:56.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.113+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5404067450 con 0x7f54040d0f40 2023-11-24T06:28:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.113+0000 7f53faffd640 1 -- 172.21.15.64:0/3846630444 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f53f4079e50 con 0x7f54040d0f40 2023-11-24T06:28:56.116 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:56.116 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:28:56.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.115+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f53e806ae40 msgr2=0x7f53e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.115+0000 7f540b57a640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f53e806ae40 0x7f53e806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5400006fd0 tx=0x7f5400008040 comp rx=0 tx=0).stop 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.115+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0f40 msgr2=0x7f54040ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.115+0000 7f540b57a640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0f40 0x7f54040ccd20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009870 tx=0x7f53f40046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 shutdown_connections 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f53e806ae40 0x7f53e806d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5404160920 0x7f540415ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54040d0f40 0x7f54040ccd20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 --2- 172.21.15.64:0/3846630444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54040d0100 0x7f540414d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 >> 172.21.15.64:0/3846630444 conn(0x7f54040d3090 msgr2=0x7f54040cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 shutdown_connections 2023-11-24T06:28:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.116+0000 7f540b57a640 1 -- 172.21.15.64:0/3846630444 wait complete. 2023-11-24T06:28:56.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.350+0000 7f5d6e1fa640 1 Processor -- start 2023-11-24T06:28:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.351+0000 7f5d6e1fa640 1 -- start start 2023-11-24T06:28:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.352+0000 7f5d6e1fa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5d6813cfa0 0x7f5d6815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.352+0000 7f5d6e1fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d680cd530 0x7f5d6813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.352+0000 7f5d6e1fa640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d680cdec0 con 0x7f5d680cd9b0 2023-11-24T06:28:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.352+0000 7f5d6e1fa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d680ce1a0 con 0x7f5d6813cfa0 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.352+0000 7f5d6e1fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680ce480 con 0x7f5d680cd530 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.353+0000 7f5d6d9f9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5d680cd9b0 0x7f5d6815d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39076/0 (socket says 172.21.15.64:39076) 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.353+0000 7f5d6d9f9640 1 -- 172.21.15.64:0/357863593 learned_addr learned my addr 172.21.15.64:0/357863593 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.353+0000 7f5d6c9f7640 1 --2- 172.21.15.64:0/357863593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d680cd530 0x7f5d6813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.353+0000 7f5d6d1f8640 1 --2- 172.21.15.64:0/357863593 >> v2:172.21.15.162:3300/0 conn(0x7f5d6813cfa0 0x7f5d6815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.354+0000 7f5d6c9f7640 1 -- 172.21.15.64:0/357863593 >> v2:172.21.15.162:3300/0 conn(0x7f5d6813cfa0 msgr2=0x7f5d6815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.354+0000 7f5d6c9f7640 1 --2- 172.21.15.64:0/357863593 >> v2:172.21.15.162:3300/0 conn(0x7f5d6813cfa0 0x7f5d6815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.354+0000 7f5d6c9f7640 1 -- 172.21.15.64:0/357863593 >> v1:172.21.15.64:6789/0 conn(0x7f5d680cd9b0 legacy=0x7f5d6815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.354+0000 7f5d5ffff640 1 -- 172.21.15.64:0/357863593 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945775221 0 0) 0x7f5d680cdec0 con 0x7f5d680cd9b0 2023-11-24T06:28:56.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.354+0000 7f5d6c9f7640 1 -- 172.21.15.64:0/357863593 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680ce760 con 0x7f5d680cd530 2023-11-24T06:28:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.355+0000 7f5d6c9f7640 1 --2- 172.21.15.64:0/357863593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d680cd530 0x7f5d6813f5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097a0 tx=0x7f5d5802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6da652c4ce7074aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d5ffff640 1 -- 172.21.15.64:0/357863593 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5802e070 con 0x7f5d680cd530 2023-11-24T06:28:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d5ffff640 1 -- 172.21.15.64:0/357863593 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d580315a0 con 0x7f5d680cd530 2023-11-24T06:28:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d5ffff640 1 -- 172.21.15.64:0/357863593 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58031930 con 0x7f5d680cd530 2023-11-24T06:28:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/357863593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d680cd530 msgr2=0x7f5d6813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/357863593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d680cd530 0x7f5d6813f5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097a0 tx=0x7f5d5802b840 comp rx=0 tx=0).stop 2023-11-24T06:28:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/357863593 shutdown_connections 2023-11-24T06:28:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/357863593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d680cd530 0x7f5d6813f5a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/357863593 >> v2:172.21.15.162:3300/0 conn(0x7f5d6813cfa0 0x7f5d6815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.356+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/357863593 >> 172.21.15.64:0/357863593 conn(0x7f5d6806a800 msgr2=0x7f5d6806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:56.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.357+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/357863593 shutdown_connections 2023-11-24T06:28:56.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.357+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/357863593 wait complete. 2023-11-24T06:28:56.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.358+0000 7f5d6e1fa640 1 Processor -- start 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.359+0000 7f5d6e1fa640 1 -- start start 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6e1fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d680cd530 0x7f5d681385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6e1fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d680cd9b0 0x7f5d680715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6e1fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d6813cfa0 0x7f5d68071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6e1fa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68150bb0 con 0x7f5d680cd530 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6e1fa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6814ff30 con 0x7f5d680cd9b0 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6e1fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68150210 con 0x7f5d6813cfa0 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6d1f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d680cd530 0x7f5d681385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6d1f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d680cd530 0x7f5d681385b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50658/0 (socket says 172.21.15.64:50658) 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6d1f8640 1 -- 172.21.15.64:0/2055153247 learned_addr learned my addr 172.21.15.64:0/2055153247 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.360+0000 7f5d6d9f9640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d6813cfa0 0x7f5d68071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6c9f7640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d680cd9b0 0x7f5d680715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6d1f8640 1 -- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d6813cfa0 msgr2=0x7f5d68071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6d1f8640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d6813cfa0 0x7f5d68071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6d1f8640 1 -- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d680cd9b0 msgr2=0x7f5d680715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6d1f8640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d680cd9b0 0x7f5d680715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6d1f8640 1 -- 172.21.15.64:0/2055153247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d58009400 con 0x7f5d680cd530 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6c9f7640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d680cd9b0 0x7f5d680715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.361+0000 7f5d6d1f8640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d680cd530 0x7f5d681385b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400e9b0 tx=0x7f5d6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.362+0000 7f5d5dffb640 1 -- 172.21.15.64:0/2055153247 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d64009670 con 0x7f5d680cd530 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.362+0000 7f5d5dffb640 1 -- 172.21.15.64:0/2055153247 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6400c9a0 con 0x7f5d680cd530 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.362+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d681504f0 con 0x7f5d680cd530 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.362+0000 7f5d5dffb640 1 -- 172.21.15.64:0/2055153247 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400cd50 con 0x7f5d680cd530 2023-11-24T06:28:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.362+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d68138af0 con 0x7f5d680cd530 2023-11-24T06:28:56.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.363+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d68067450 con 0x7f5d680cd530 2023-11-24T06:28:56.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.365+0000 7f5d5dffb640 1 -- 172.21.15.64:0/2055153247 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d64013470 con 0x7f5d680cd530 2023-11-24T06:28:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.366+0000 7f5d5dffb640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5d4c06ae40 0x7f5d4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.366+0000 7f5d5dffb640 1 -- 172.21.15.64:0/2055153247 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f5d64089830 con 0x7f5d680cd530 2023-11-24T06:28:56.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.367+0000 7f5d6c9f7640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5d4c06ae40 0x7f5d4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.367+0000 7f5d6c9f7640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5d4c06ae40 0x7f5d4c06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d58036000 tx=0x7f5d5803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.375+0000 7f5d5dffb640 1 -- 172.21.15.64:0/2055153247 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d64054910 con 0x7f5d680cd530 2023-11-24T06:28:56.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.725+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d680b4000 con 0x7f5d680cd530 2023-11-24T06:28:56.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.726+0000 7f5d5dffb640 1 -- 172.21.15.64:0/2055153247 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f5d64017090 con 0x7f5d680cd530 2023-11-24T06:28:56.729 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:56.729 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3287102658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3287102658},{"type":"v1","addr":"172.21.15.162:6835","nonce":3287102658}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1476221336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1476221336},{"type":"v1","addr":"172.21.15.64:6835","nonce":1476221336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4227,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/866855163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":866855163},{"type":"v1","addr":"172.21.15.64:6837","nonce":866855163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:41.828633+0000","modified":"2023-11-24T06:27:45.282437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4246":{"gid":4246,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.162:6833/3583074090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3583074090},{"type":"v1","addr":"172.21.15.162:6833","nonce":3583074090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:56.730 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T06:28:56.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5d4c06ae40 msgr2=0x7f5d4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5d4c06ae40 0x7f5d4c06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d58036000 tx=0x7f5d5803d040 comp rx=0 tx=0).stop 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d680cd530 msgr2=0x7f5d681385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d680cd530 0x7f5d681385b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400e9b0 tx=0x7f5d6400ee70 comp rx=0 tx=0).stop 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 shutdown_connections 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5d4c06ae40 0x7f5d4c06d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d6813cfa0 0x7f5d68071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d680cd9b0 0x7f5d680715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 --2- 172.21.15.64:0/2055153247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d680cd530 0x7f5d681385b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 >> 172.21.15.64:0/2055153247 conn(0x7f5d6806a800 msgr2=0x7f5d68158820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 shutdown_connections 2023-11-24T06:28:56.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:56.729+0000 7f5d6e1fa640 1 -- 172.21.15.64:0/2055153247 wait complete. 2023-11-24T06:28:56.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:57.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.008+0000 7f0ad410f640 1 Processor -- start 2023-11-24T06:28:57.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.009+0000 7f0ad410f640 1 -- start start 2023-11-24T06:28:57.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad410f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0acc06a3e0 0x7f0acc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad410f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0acc0aede0 0x7f0acc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad410f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0acc071de0 con 0x7f0acc0d0b60 2023-11-24T06:28:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad410f640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0acc0af270 con 0x7f0acc0aede0 2023-11-24T06:28:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad410f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0acc072030 con 0x7f0acc06a3e0 2023-11-24T06:28:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad26ac640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0acc0aede0 0x7f0acc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad16aa640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0acc0d0b60 0x7f0acc06a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:46216/0 (socket says 172.21.15.64:46216) 2023-11-24T06:28:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.010+0000 7f0ad16aa640 1 -- 172.21.15.64:0/2456637432 learned_addr learned my addr 172.21.15.64:0/2456637432 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:57.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.011+0000 7f0ad1eab640 1 --2- 172.21.15.64:0/2456637432 >> v2:172.21.15.162:3300/0 conn(0x7f0acc06a3e0 0x7f0acc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.011+0000 7f0ad0ea9640 1 -- 172.21.15.64:0/2456637432 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427964527 0 0) 0x7f0acc071de0 con 0x7f0acc0d0b60 2023-11-24T06:28:57.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.011+0000 7f0ad0ea9640 1 -- 172.21.15.64:0/2456637432 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab8003520 con 0x7f0acc0d0b60 2023-11-24T06:28:57.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.011+0000 7f0ad26ac640 1 -- 172.21.15.64:0/2456637432 >> v2:172.21.15.162:3300/0 conn(0x7f0acc06a3e0 msgr2=0x7f0acc0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:57.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.011+0000 7f0ad26ac640 1 --2- 172.21.15.64:0/2456637432 >> v2:172.21.15.162:3300/0 conn(0x7f0acc06a3e0 0x7f0acc0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.011+0000 7f0ad26ac640 1 -- 172.21.15.64:0/2456637432 >> v1:172.21.15.162:6789/0 conn(0x7f0acc0d0b60 legacy=0x7f0acc06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.011+0000 7f0ad26ac640 1 -- 172.21.15.64:0/2456637432 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc072310 con 0x7f0acc0aede0 2023-11-24T06:28:57.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.012+0000 7f0ad26ac640 1 --2- 172.21.15.64:0/2456637432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0aede0 0x7f0acc06f080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0acc073fe0 tx=0x7f0ac402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a4cf6503c4f7ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.013+0000 7f0ad0ea9640 1 -- 172.21.15.64:0/2456637432 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac402cc80 con 0x7f0acc0aede0 2023-11-24T06:28:57.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.013+0000 7f0ad0ea9640 1 -- 172.21.15.64:0/2456637432 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac402cde0 con 0x7f0acc0aede0 2023-11-24T06:28:57.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.013+0000 7f0ad0ea9640 1 -- 172.21.15.64:0/2456637432 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4034660 con 0x7f0acc0aede0 2023-11-24T06:28:57.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.013+0000 7f0ad410f640 1 -- 172.21.15.64:0/2456637432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0aede0 msgr2=0x7f0acc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.013+0000 7f0ad410f640 1 --2- 172.21.15.64:0/2456637432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0aede0 0x7f0acc06f080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0acc073fe0 tx=0x7f0ac402bed0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.014+0000 7f0ad410f640 1 -- 172.21.15.64:0/2456637432 shutdown_connections 2023-11-24T06:28:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.014+0000 7f0ad410f640 1 --2- 172.21.15.64:0/2456637432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0aede0 0x7f0acc06f080 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.014+0000 7f0ad410f640 1 --2- 172.21.15.64:0/2456637432 >> v2:172.21.15.162:3300/0 conn(0x7f0acc06a3e0 0x7f0acc0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.014+0000 7f0ad410f640 1 -- 172.21.15.64:0/2456637432 >> 172.21.15.64:0/2456637432 conn(0x7f0acc0cfef0 msgr2=0x7f0acc0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.014+0000 7f0ad410f640 1 -- 172.21.15.64:0/2456637432 shutdown_connections 2023-11-24T06:28:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.014+0000 7f0ad410f640 1 -- 172.21.15.64:0/2456637432 wait complete. 2023-11-24T06:28:57.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.016+0000 7f0ad410f640 1 Processor -- start 2023-11-24T06:28:57.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.017+0000 7f0ad410f640 1 -- start start 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad410f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0acc06a3e0 0x7f0acc0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad410f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0acc0aede0 0x7f0acc1f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad410f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0d0b60 0x7f0acc1f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad410f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc145420 con 0x7f0acc0d0b60 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad410f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc15b160 con 0x7f0acc0aede0 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad410f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc15b370 con 0x7f0acc06a3e0 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad16aa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0acc0aede0 0x7f0acc1f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad1eab640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0acc06a3e0 0x7f0acc0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad26ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0d0b60 0x7f0acc1f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad16aa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0acc0aede0 0x7f0acc1f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:43914/0 (socket says 172.21.15.64:43914) 2023-11-24T06:28:57.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.018+0000 7f0ad16aa640 1 -- 172.21.15.64:0/1520048680 learned_addr learned my addr 172.21.15.64:0/1520048680 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:57.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.019+0000 7f0ad26ac640 1 -- 172.21.15.64:0/1520048680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0acc06a3e0 msgr2=0x7f0acc0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.019+0000 7f0ad26ac640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0acc06a3e0 0x7f0acc0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.019+0000 7f0ad26ac640 1 -- 172.21.15.64:0/1520048680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0acc0aede0 msgr2=0x7f0acc1f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.019+0000 7f0ad26ac640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0acc0aede0 0x7f0acc1f5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.019+0000 7f0ad26ac640 1 -- 172.21.15.64:0/1520048680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4009a70 con 0x7f0acc0d0b60 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.019+0000 7f0ad26ac640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0d0b60 0x7f0acc1f6360 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4033000 tx=0x7f0ac4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.019+0000 7f0ac27fc640 1 -- 172.21.15.64:0/1520048680 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4009450 con 0x7f0acc0d0b60 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.020+0000 7f0ac27fc640 1 -- 172.21.15.64:0/1520048680 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac4040450 con 0x7f0acc0d0b60 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.020+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0acc15b650 con 0x7f0acc0d0b60 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.020+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0acc15bc10 con 0x7f0acc0d0b60 2023-11-24T06:28:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.020+0000 7f0ac27fc640 1 -- 172.21.15.64:0/1520048680 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40407c0 con 0x7f0acc0d0b60 2023-11-24T06:28:57.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.021+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0acc0675d0 con 0x7f0acc0d0b60 2023-11-24T06:28:57.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.023+0000 7f0ac27fc640 1 -- 172.21.15.64:0/1520048680 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ac4040920 con 0x7f0acc0d0b60 2023-11-24T06:28:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.024+0000 7f0ac27fc640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0ab006ae40 0x7f0ab006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.024+0000 7f0ac27fc640 1 -- 172.21.15.64:0/1520048680 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f0ac40ac180 con 0x7f0acc0d0b60 2023-11-24T06:28:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.024+0000 7f0ad1eab640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0ab006ae40 0x7f0ab006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.025+0000 7f0ad1eab640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0ab006ae40 0x7f0ab006d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009aa0 tx=0x7f0ac8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.033+0000 7f0ac27fc640 1 -- 172.21.15.64:0/1520048680 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ac40771e0 con 0x7f0acc0d0b60 2023-11-24T06:28:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.248+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0acc0b3ee0 con 0x7f0acc0d0b60 2023-11-24T06:28:57.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.250+0000 7f0ac27fc640 1 -- 172.21.15.64:0/1520048680 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f0ac407a800 con 0x7f0acc0d0b60 2023-11-24T06:28:57.253 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:28:57.253 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:28:29.007980+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:39.545535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:40.829254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0ab006ae40 msgr2=0x7f0ab006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0ab006ae40 0x7f0ab006d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009aa0 tx=0x7f0ac8009190 comp rx=0 tx=0).stop 2023-11-24T06:28:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0d0b60 msgr2=0x7f0acc1f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0d0b60 0x7f0acc1f6360 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4033000 tx=0x7f0ac4004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 shutdown_connections 2023-11-24T06:28:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0ab006ae40 0x7f0ab006d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc0d0b60 0x7f0acc1f6360 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0acc0aede0 0x7f0acc1f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 --2- 172.21.15.64:0/1520048680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0acc06a3e0 0x7f0acc0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 >> 172.21.15.64:0/1520048680 conn(0x7f0acc0cfef0 msgr2=0x7f0acc0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 shutdown_connections 2023-11-24T06:28:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.252+0000 7f0ad410f640 1 -- 172.21.15.64:0/1520048680 wait complete. 2023-11-24T06:28:57.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:28:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.502+0000 7f9dcf516640 1 Processor -- start 2023-11-24T06:28:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.502+0000 7f9dcf516640 1 -- start start 2023-11-24T06:28:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dcf516640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9dc8160920 0x7f9dc80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dcf516640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9dc80cfea0 0x7f9dc8161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dcf516640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80d0a10 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dcf516640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80d1f50 con 0x7f9dc8160920 2023-11-24T06:28:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dcf516640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80cf080 con 0x7f9dc80cfea0 2023-11-24T06:28:57.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dce514640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9dc8160920 0x7f9dc80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dce514640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9dc8160920 0x7f9dc80b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50692/0 (socket says 172.21.15.64:50692) 2023-11-24T06:28:57.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dcdd13640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9dc80cf9f0 0x7f9dc8161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39088/0 (socket says 172.21.15.64:39088) 2023-11-24T06:28:57.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dce514640 1 -- 172.21.15.64:0/3331838923 learned_addr learned my addr 172.21.15.64:0/3331838923 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:57.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.504+0000 7f9dced15640 1 --2- 172.21.15.64:0/3331838923 >> v2:172.21.15.162:3300/0 conn(0x7f9dc80cfea0 0x7f9dc8161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.505+0000 7f9dce514640 1 -- 172.21.15.64:0/3331838923 >> v2:172.21.15.162:3300/0 conn(0x7f9dc80cfea0 msgr2=0x7f9dc8161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:57.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.505+0000 7f9dce514640 1 --2- 172.21.15.64:0/3331838923 >> v2:172.21.15.162:3300/0 conn(0x7f9dc80cfea0 0x7f9dc8161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.505+0000 7f9dcd512640 1 -- 172.21.15.64:0/3331838923 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282729104 0 0) 0x7f9dc80d0a10 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.505+0000 7f9dce514640 1 -- 172.21.15.64:0/3331838923 >> v1:172.21.15.64:6789/0 conn(0x7f9dc80cf9f0 legacy=0x7f9dc8161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.505+0000 7f9dce514640 1 -- 172.21.15.64:0/3331838923 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80d0310 con 0x7f9dc8160920 2023-11-24T06:28:57.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.505+0000 7f9dced15640 1 --2- 172.21.15.64:0/3331838923 >> v2:172.21.15.162:3300/0 conn(0x7f9dc80cfea0 0x7f9dc8161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:57.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.506+0000 7f9dce514640 1 --2- 172.21.15.64:0/3331838923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc8160920 0x7f9dc80b3f30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9dc808c2c0 tx=0x7f9dc402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2055cadd3f9d660 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.507+0000 7f9dcd512640 1 -- 172.21.15.64:0/3331838923 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4030860 con 0x7f9dc8160920 2023-11-24T06:28:57.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.507+0000 7f9dcd512640 1 -- 172.21.15.64:0/3331838923 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40309c0 con 0x7f9dc8160920 2023-11-24T06:28:57.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.507+0000 7f9dcd512640 1 -- 172.21.15.64:0/3331838923 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4030d50 con 0x7f9dc8160920 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 -- 172.21.15.64:0/3331838923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc8160920 msgr2=0x7f9dc80b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 --2- 172.21.15.64:0/3331838923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc8160920 0x7f9dc80b3f30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9dc808c2c0 tx=0x7f9dc402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 -- 172.21.15.64:0/3331838923 shutdown_connections 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 --2- 172.21.15.64:0/3331838923 >> v2:172.21.15.162:3300/0 conn(0x7f9dc80cfea0 0x7f9dc8161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 --2- 172.21.15.64:0/3331838923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc8160920 0x7f9dc80b3f30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 -- 172.21.15.64:0/3331838923 >> 172.21.15.64:0/3331838923 conn(0x7f9dc80d3090 msgr2=0x7f9dc80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 -- 172.21.15.64:0/3331838923 shutdown_connections 2023-11-24T06:28:57.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.508+0000 7f9dcf516640 1 -- 172.21.15.64:0/3331838923 wait complete. 2023-11-24T06:28:57.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.510+0000 7f9dcf516640 1 Processor -- start 2023-11-24T06:28:57.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.510+0000 7f9dcf516640 1 -- start start 2023-11-24T06:28:57.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.511+0000 7f9dcf516640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc80cf9f0 0x7f9dc81546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.511+0000 7f9dcf516640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc80cfea0 0x7f9dc80cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.511+0000 7f9dcf516640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc8160920 0x7f9dc80b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.511+0000 7f9dcf516640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc8154c10 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.511+0000 7f9dcf516640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80b7830 con 0x7f9dc80cfea0 2023-11-24T06:28:57.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.511+0000 7f9dcf516640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80ca160 con 0x7f9dc8160920 2023-11-24T06:28:57.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc80cf9f0 0x7f9dc81546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc80cf9f0 0x7f9dc81546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50704/0 (socket says 172.21.15.64:50704) 2023-11-24T06:28:57.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 -- 172.21.15.64:0/1344604472 learned_addr learned my addr 172.21.15.64:0/1344604472 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:57.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dced15640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc8160920 0x7f9dc80b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dcdd13640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc80cfea0 0x7f9dc80cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 -- 172.21.15.64:0/1344604472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc8160920 msgr2=0x7f9dc80b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc8160920 0x7f9dc80b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 -- 172.21.15.64:0/1344604472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc80cfea0 msgr2=0x7f9dc80cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc80cfea0 0x7f9dc80cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.512+0000 7f9dce514640 1 -- 172.21.15.64:0/1344604472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc40094d0 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.513+0000 7f9dce514640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc80cf9f0 0x7f9dc81546d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9dc402b9e0 tx=0x7f9dc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.513+0000 7f9dbeffd640 1 -- 172.21.15.64:0/1344604472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4039dd0 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.513+0000 7f9dbeffd640 1 -- 172.21.15.64:0/1344604472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4043400 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.513+0000 7f9dbeffd640 1 -- 172.21.15.64:0/1344604472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40437b0 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.513+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc80cb0f0 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.514+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc80b7550 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.515+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc8067450 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.516+0000 7f9dbeffd640 1 -- 172.21.15.64:0/1344604472 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9dc4043910 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.517+0000 7f9dbeffd640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dac06ae70 0x7f9dac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.517+0000 7f9dbeffd640 1 -- 172.21.15.64:0/1344604472 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f9dc403e070 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.518+0000 7f9dcdd13640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dac06ae70 0x7f9dac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.519+0000 7f9dcdd13640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dac06ae70 0x7f9dac06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9dc81f0bb0 tx=0x7f9db8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.521+0000 7f9dbeffd640 1 -- 172.21.15.64:0/1344604472 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9dc40782a0 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.911+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9dc808c7e0 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.915 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T06:28:57.912+0000 7fa0b5f4b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:28:57.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.916+0000 7f9dbeffd640 1 -- 172.21.15.64:0/1344604472 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9dc407b8c0 con 0x7f9dc80cf9f0 2023-11-24T06:28:57.920 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:28:57.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dac06ae70 msgr2=0x7f9dac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dac06ae70 0x7f9dac06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9dc81f0bb0 tx=0x7f9db8007de0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc80cf9f0 msgr2=0x7f9dc81546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc80cf9f0 0x7f9dc81546d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9dc402b9e0 tx=0x7f9dc4004730 comp rx=0 tx=0).stop 2023-11-24T06:28:57.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 shutdown_connections 2023-11-24T06:28:57.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dac06ae70 0x7f9dac06d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc8160920 0x7f9dc80b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc80cfea0 0x7f9dc80cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 --2- 172.21.15.64:0/1344604472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc80cf9f0 0x7f9dc81546d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.919+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 >> 172.21.15.64:0/1344604472 conn(0x7f9dc80d3090 msgr2=0x7f9dc815b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:57.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.920+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 shutdown_connections 2023-11-24T06:28:57.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:57.920+0000 7f9dcf516640 1 -- 172.21.15.64:0/1344604472 wait complete. 2023-11-24T06:28:57.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:28:58.004 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2023-11-24T06:28:58.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.190+0000 7f0227fff640 1 Processor -- start 2023-11-24T06:28:58.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.190+0000 7f0227fff640 1 -- start start 2023-11-24T06:28:58.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.192+0000 7f0227fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f02280adf20 0x7f0228070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.192+0000 7f0227fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f022815bf10 0x7f02280cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.192+0000 7f0227fff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022815f4e0 con 0x7f02280cc980 2023-11-24T06:28:58.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.192+0000 7f0227fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022815f970 con 0x7f02280adf20 2023-11-24T06:28:58.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.192+0000 7f0227fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cce10 con 0x7f022815bf10 2023-11-24T06:28:58.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.193+0000 7f02277fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f02280cc980 0x7f0228143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:46218/0 (socket says 172.21.15.64:46218) 2023-11-24T06:28:58.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.193+0000 7f02277fe640 1 -- 172.21.15.64:0/4154091486 learned_addr learned my addr 172.21.15.64:0/4154091486 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:58.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.193+0000 7f0226ffd640 1 --2- 172.21.15.64:0/4154091486 >> v2:172.21.15.162:3300/0 conn(0x7f02280adf20 0x7f0228070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.193+0000 7f02267fc640 1 --2- 172.21.15.64:0/4154091486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f022815bf10 0x7f02280cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.194+0000 7f0225ffb640 1 -- 172.21.15.64:0/4154091486 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216050893 0 0) 0x7f022815f4e0 con 0x7f02280cc980 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.194+0000 7f0225ffb640 1 -- 172.21.15.64:0/4154091486 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0210003520 con 0x7f02280cc980 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.194+0000 7f02267fc640 1 -- 172.21.15.64:0/4154091486 >> v2:172.21.15.162:3300/0 conn(0x7f02280adf20 msgr2=0x7f0228070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.194+0000 7f02267fc640 1 --2- 172.21.15.64:0/4154091486 >> v2:172.21.15.162:3300/0 conn(0x7f02280adf20 0x7f0228070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.194+0000 7f02267fc640 1 -- 172.21.15.64:0/4154091486 >> v1:172.21.15.162:6789/0 conn(0x7f02280cc980 legacy=0x7f0228143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.194+0000 7f02267fc640 1 -- 172.21.15.64:0/4154091486 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0228071130 con 0x7f022815bf10 2023-11-24T06:28:58.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.195+0000 7f02267fc640 1 --2- 172.21.15.64:0/4154091486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f022815bf10 0x7f02280cc440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f021c0097a0 tx=0x7f021c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae7b56e18d2bd2a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:58.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.195+0000 7f0225ffb640 1 -- 172.21.15.64:0/4154091486 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c031780 con 0x7f022815bf10 2023-11-24T06:28:58.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.196+0000 7f0225ffb640 1 -- 172.21.15.64:0/4154091486 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c0318e0 con 0x7f022815bf10 2023-11-24T06:28:58.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.196+0000 7f0225ffb640 1 -- 172.21.15.64:0/4154091486 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c031c50 con 0x7f022815bf10 2023-11-24T06:28:58.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.196+0000 7f0227fff640 1 -- 172.21.15.64:0/4154091486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f022815bf10 msgr2=0x7f02280cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.196+0000 7f0227fff640 1 --2- 172.21.15.64:0/4154091486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f022815bf10 0x7f02280cc440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f021c0097a0 tx=0x7f021c02b820 comp rx=0 tx=0).stop 2023-11-24T06:28:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.197+0000 7f0227fff640 1 -- 172.21.15.64:0/4154091486 shutdown_connections 2023-11-24T06:28:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.197+0000 7f0227fff640 1 --2- 172.21.15.64:0/4154091486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f022815bf10 0x7f02280cc440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.197+0000 7f0227fff640 1 --2- 172.21.15.64:0/4154091486 >> v2:172.21.15.162:3300/0 conn(0x7f02280adf20 0x7f0228070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.197+0000 7f0227fff640 1 -- 172.21.15.64:0/4154091486 >> 172.21.15.64:0/4154091486 conn(0x7f022806a7d0 msgr2=0x7f022806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.198+0000 7f0227fff640 1 -- 172.21.15.64:0/4154091486 shutdown_connections 2023-11-24T06:28:58.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.198+0000 7f0227fff640 1 -- 172.21.15.64:0/4154091486 wait complete. 2023-11-24T06:28:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.199+0000 7f0227fff640 1 Processor -- start 2023-11-24T06:28:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.199+0000 7f0227fff640 1 -- start start 2023-11-24T06:28:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.200+0000 7f0227fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02280adf20 0x7f022813a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.200+0000 7f0227fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02280cc980 0x7f02280c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.200+0000 7f0227fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f022815bf10 0x7f02280c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.200+0000 7f0227fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022813a9a0 con 0x7f022815bf10 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.200+0000 7f0227fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cada0 con 0x7f02280adf20 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.200+0000 7f0227fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280ca190 con 0x7f02280cc980 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f0226ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02280adf20 0x7f022813a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f02267fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02280cc980 0x7f02280c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f02277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f022815bf10 0x7f02280c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f0226ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02280adf20 0x7f022813a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:43946/0 (socket says 172.21.15.64:43946) 2023-11-24T06:28:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f0226ffd640 1 -- 172.21.15.64:0/3568756582 learned_addr learned my addr 172.21.15.64:0/3568756582 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f02277fe640 1 -- 172.21.15.64:0/3568756582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02280cc980 msgr2=0x7f02280c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f02277fe640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02280cc980 0x7f02280c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f02277fe640 1 -- 172.21.15.64:0/3568756582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02280adf20 msgr2=0x7f022813a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f02277fe640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02280adf20 0x7f022813a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.201+0000 7f02277fe640 1 -- 172.21.15.64:0/3568756582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c009400 con 0x7f022815bf10 2023-11-24T06:28:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.202+0000 7f02277fe640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f022815bf10 0x7f02280c9b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f022000ca80 tx=0x7f022000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:58.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.202+0000 7f020ffff640 1 -- 172.21.15.64:0/3568756582 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022000ac30 con 0x7f022815bf10 2023-11-24T06:28:58.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.202+0000 7f020ffff640 1 -- 172.21.15.64:0/3568756582 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022000ad90 con 0x7f022815bf10 2023-11-24T06:28:58.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.202+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02280ca470 con 0x7f022815bf10 2023-11-24T06:28:58.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.202+0000 7f020ffff640 1 -- 172.21.15.64:0/3568756582 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220012600 con 0x7f022815bf10 2023-11-24T06:28:58.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.202+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02280caa30 con 0x7f022815bf10 2023-11-24T06:28:58.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.204+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02280caf10 con 0x7f022815bf10 2023-11-24T06:28:58.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.205+0000 7f020ffff640 1 -- 172.21.15.64:0/3568756582 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0220007070 con 0x7f022815bf10 2023-11-24T06:28:58.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.206+0000 7f020ffff640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f021406ae90 0x7f021406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.206+0000 7f020ffff640 1 -- 172.21.15.64:0/3568756582 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f0220089580 con 0x7f022815bf10 2023-11-24T06:28:58.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.207+0000 7f0226ffd640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f021406ae90 0x7f021406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.208+0000 7f0226ffd640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f021406ae90 0x7f021406d350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0218009b60 tx=0x7f0218009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:58.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.212+0000 7f020ffff640 1 -- 172.21.15.64:0/3568756582 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0220054600 con 0x7f022815bf10 2023-11-24T06:28:58.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.576+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0228067450 con 0x7f022815bf10 2023-11-24T06:28:58.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.918+0000 7f020ffff640 1 -- 172.21.15.64:0/3568756582 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0220057c20 con 0x7f022815bf10 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f021406ae90 msgr2=0x7f021406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f021406ae90 0x7f021406d350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0218009b60 tx=0x7f0218009190 comp rx=0 tx=0).stop 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f022815bf10 msgr2=0x7f02280c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f022815bf10 0x7f02280c9b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f022000ca80 tx=0x7f022000cf40 comp rx=0 tx=0).stop 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 shutdown_connections 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f021406ae90 0x7f021406d350 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f022815bf10 0x7f02280c9b30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02280cc980 0x7f02280c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 --2- 172.21.15.64:0/3568756582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02280adf20 0x7f022813a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 >> 172.21.15.64:0/3568756582 conn(0x7f022806a7d0 msgr2=0x7f02280cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.921+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 shutdown_connections 2023-11-24T06:28:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:58.922+0000 7f0227fff640 1 -- 172.21.15.64:0/3568756582 wait complete. 2023-11-24T06:28:59.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:28:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.404+0000 7f99728df640 1 Processor -- start 2023-11-24T06:28:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.404+0000 7f99728df640 1 -- start start 2023-11-24T06:28:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.405+0000 7f99728df640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f996c146850 0x7f996c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.405+0000 7f99728df640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 0x7f996c0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.405+0000 7f99728df640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c06d480 con 0x7f996c0adb00 2023-11-24T06:28:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.405+0000 7f99728df640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c071fa0 con 0x7f996c146850 2023-11-24T06:28:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.405+0000 7f99728df640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0b3490 con 0x7f996c071b10 2023-11-24T06:28:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.406+0000 7f99720de640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f996c0adb00 0x7f996c0700f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39090/0 (socket says 172.21.15.64:39090) 2023-11-24T06:28:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.406+0000 7f99720de640 1 -- 172.21.15.64:0/3366890923 learned_addr learned my addr 172.21.15.64:0/3366890923 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.406+0000 7f99710dc640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 0x7f996c0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.406+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969634436 0 0) 0x7f996c06d480 con 0x7f996c0adb00 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.406+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9958003520 con 0x7f996c0adb00 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.406+0000 7f99718dd640 1 --2- 172.21.15.64:0/3366890923 >> v2:172.21.15.162:3300/0 conn(0x7f996c146850 0x7f996c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1884616837 0 0) 0x7f9958003520 con 0x7f996c0adb00 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 msgr2=0x7f996c0ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 0x7f996c0ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 >> v2:172.21.15.162:3300/0 conn(0x7f996c146850 msgr2=0x7f996c0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 --2- 172.21.15.64:0/3366890923 >> v2:172.21.15.162:3300/0 conn(0x7f996c146850 0x7f996c0715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0b3770 con 0x7f996c0adb00 2023-11-24T06:28:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f99640019f0 con 0x7f996c0adb00 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99710dc640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 0x7f996c0ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 >> v1:172.21.15.64:6789/0 conn(0x7f996c0adb00 legacy=0x7f996c0700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9958003970 0x7f9958003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99580047c0 0x7f9958004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 0x7f9958005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c071fa0 con 0x7f9958003970 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0b3490 con 0x7f99580047c0 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995800ba30 con 0x7f996c071b10 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99718dd640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9958003970 0x7f9958003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99718dd640 1 -- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 msgr2=0x7f9958005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.407+0000 7f99718dd640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c071b10 0x7f9958005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.408+0000 7f99710dc640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99580047c0 0x7f9958004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.408+0000 7f99718dd640 1 -- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99580047c0 msgr2=0x7f9958004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.408+0000 7f99718dd640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99580047c0 0x7f9958004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.408+0000 7f99718dd640 1 -- 172.21.15.64:0/3366890923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958003520 con 0x7f9958003970 2023-11-24T06:28:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.409+0000 7f99718dd640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9958003970 0x7f9958003d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f996800ba30 tx=0x7f99580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f5c8f026e13e6e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.409+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968032680 con 0x7f9958003970 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.409+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99680327e0 con 0x7f9958003970 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.409+0000 7f99708db640 1 -- 172.21.15.64:0/3366890923 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968032b70 con 0x7f9958003970 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.409+0000 7f99728df640 1 -- 172.21.15.64:0/3366890923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9958003970 msgr2=0x7f9958003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.409+0000 7f99728df640 1 --2- 172.21.15.64:0/3366890923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9958003970 0x7f9958003d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f996800ba30 tx=0x7f99580057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.409+0000 7f99720de640 1 -- 172.21.15.64:0/3366890923 reap_dead start 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.410+0000 7f99728df640 1 -- 172.21.15.64:0/3366890923 shutdown_connections 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.410+0000 7f99728df640 1 -- 172.21.15.64:0/3366890923 >> 172.21.15.64:0/3366890923 conn(0x7f996c06a3e0 msgr2=0x7f996c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.410+0000 7f99728df640 1 -- 172.21.15.64:0/3366890923 shutdown_connections 2023-11-24T06:28:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.410+0000 7f99728df640 1 -- 172.21.15.64:0/3366890923 wait complete. 2023-11-24T06:28:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.411+0000 7f99728df640 1 Processor -- start 2023-11-24T06:28:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.412+0000 7f99728df640 1 -- start start 2023-11-24T06:28:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.412+0000 7f99728df640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9958003970 0x7f996c138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99728df640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c0c6510 0x7f996c0c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99728df640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f996c0c47c0 0x7f996c0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99728df640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c138ea0 con 0x7f996c0c47c0 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99728df640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0c6eb0 con 0x7f9958003970 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99728df640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0c5470 con 0x7f996c0c6510 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f996c0c47c0 0x7f996c0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f996c0c47c0 0x7f996c0c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50742/0 (socket says 172.21.15.64:50742) 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 -- 172.21.15.64:0/3846317549 learned_addr learned my addr 172.21.15.64:0/3846317549 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99710dc640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c0c6510 0x7f996c0c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99718dd640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9958003970 0x7f996c138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 -- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c0c6510 msgr2=0x7f996c0c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c0c6510 0x7f996c0c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 -- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9958003970 msgr2=0x7f996c138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9958003970 0x7f996c138960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.413+0000 7f99720de640 1 -- 172.21.15.64:0/3846317549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0c5750 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.414+0000 7f99720de640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f996c0c47c0 0x7f996c0c4c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f996c0ad470 tx=0x7f996400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.414+0000 7f99627fc640 1 -- 172.21.15.64:0/3846317549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400a610 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.414+0000 7f99627fc640 1 -- 172.21.15.64:0/3846317549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996400a770 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.414+0000 7f99627fc640 1 -- 172.21.15.64:0/3846317549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400ab20 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.414+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c0c5a30 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.414+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c0c5ff0 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.415+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c0c61e0 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.416+0000 7f99627fc640 1 -- 172.21.15.64:0/3846317549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f996400ac80 con 0x7f996c0c47c0 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.417+0000 7f99627fc640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f995006ae70 0x7f995006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.417+0000 7f99627fc640 1 -- 172.21.15.64:0/3846317549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f9964089280 con 0x7f996c0c47c0 2023-11-24T06:28:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.417+0000 7f99718dd640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f995006ae70 0x7f995006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.418+0000 7f99718dd640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f995006ae70 0x7f995006d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f996800ba00 tx=0x7f9968030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.421+0000 7f99627fc640 1 -- 172.21.15.64:0/3846317549 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9964054270 con 0x7f996c0c47c0 2023-11-24T06:28:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:28:59.619+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f996c067450 con 0x7f996c0c47c0 2023-11-24T06:29:00.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.170+0000 7f99627fc640 1 -- 172.21.15.64:0/3846317549 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9964057890 con 0x7f996c0c47c0 2023-11-24T06:29:00.173 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:29:00.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f995006ae70 msgr2=0x7f995006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f995006ae70 0x7f995006d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f996800ba00 tx=0x7f9968030000 comp rx=0 tx=0).stop 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f996c0c47c0 msgr2=0x7f996c0c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f996c0c47c0 0x7f996c0c4c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f996c0ad470 tx=0x7f996400ccd0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 shutdown_connections 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f995006ae70 0x7f995006d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f996c0c47c0 0x7f996c0c4c40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f996c0c6510 0x7f996c0c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 --2- 172.21.15.64:0/3846317549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9958003970 0x7f996c138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 >> 172.21.15.64:0/3846317549 conn(0x7f996c06a3e0 msgr2=0x7f996c0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:00.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 shutdown_connections 2023-11-24T06:29:00.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.174+0000 7f99728df640 1 -- 172.21.15.64:0/3846317549 wait complete. 2023-11-24T06:29:00.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:29:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.413+0000 7f06d8a35640 1 Processor -- start 2023-11-24T06:29:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.413+0000 7f06d8a35640 1 -- start start 2023-11-24T06:29:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.414+0000 7f06d8a35640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f06d4143c40 0x7f06d4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.414+0000 7f06d8a35640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06d40717d0 0x7f06d40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.414+0000 7f06d8a35640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d40cca40 0x7f06d415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.414+0000 7f06d8a35640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d40723c0 con 0x7f06d40717d0 2023-11-24T06:29:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.414+0000 7f06d8a35640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d4070e50 con 0x7f06d4143c40 2023-11-24T06:29:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.414+0000 7f06d8a35640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40cce40 con 0x7f06d40cca40 2023-11-24T06:29:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06d40717d0 0x7f06d40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06d40717d0 0x7f06d40cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50754/0 (socket says 172.21.15.64:50754) 2023-11-24T06:29:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 -- 172.21.15.64:0/3561315806 learned_addr learned my addr 172.21.15.64:0/3561315806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 -- 172.21.15.64:0/3561315806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d40cca40 msgr2=0x7f06d415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 --2- 172.21.15.64:0/3561315806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d40cca40 0x7f06d415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 -- 172.21.15.64:0/3561315806 >> v2:172.21.15.162:3300/0 conn(0x7f06d4143c40 msgr2=0x7f06d4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06d2d9d640 1 --2- 172.21.15.64:0/3561315806 >> v2:172.21.15.162:3300/0 conn(0x7f06d4143c40 0x7f06d4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 --2- 172.21.15.64:0/3561315806 >> v2:172.21.15.162:3300/0 conn(0x7f06d4143c40 0x7f06d4071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.415+0000 7f06cbfff640 1 -- 172.21.15.64:0/3561315806 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4144020 con 0x7f06d40717d0 2023-11-24T06:29:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.416+0000 7f06cbfff640 1 --2- 172.21.15.64:0/3561315806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d40cc500 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f06d406e410 tx=0x7f06bc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab44825f10870954 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:00.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d259c640 1 -- 172.21.15.64:0/3561315806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0317e0 con 0x7f06d40717d0 2023-11-24T06:29:00.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d259c640 1 -- 172.21.15.64:0/3561315806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc031940 con 0x7f06d40717d0 2023-11-24T06:29:00.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d259c640 1 -- 172.21.15.64:0/3561315806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc031cd0 con 0x7f06d40717d0 2023-11-24T06:29:00.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 -- 172.21.15.64:0/3561315806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 msgr2=0x7f06d40cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 --2- 172.21.15.64:0/3561315806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d40cc500 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f06d406e410 tx=0x7f06bc02b880 comp rx=0 tx=0).stop 2023-11-24T06:29:00.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 -- 172.21.15.64:0/3561315806 shutdown_connections 2023-11-24T06:29:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 --2- 172.21.15.64:0/3561315806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d40cca40 0x7f06d415eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 --2- 172.21.15.64:0/3561315806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d40cc500 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 --2- 172.21.15.64:0/3561315806 >> v2:172.21.15.162:3300/0 conn(0x7f06d4143c40 0x7f06d4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 -- 172.21.15.64:0/3561315806 >> 172.21.15.64:0/3561315806 conn(0x7f06d406a800 msgr2=0x7f06d406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 -- 172.21.15.64:0/3561315806 shutdown_connections 2023-11-24T06:29:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.417+0000 7f06d8a35640 1 -- 172.21.15.64:0/3561315806 wait complete. 2023-11-24T06:29:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.419+0000 7f06d8a35640 1 Processor -- start 2023-11-24T06:29:00.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.419+0000 7f06d8a35640 1 -- start start 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.420+0000 7f06d8a35640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.420+0000 7f06d8a35640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06d40cca40 0x7f06d40ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.420+0000 7f06d8a35640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d4143c40 0x7f06d41e8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.420+0000 7f06d8a35640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c5950 con 0x7f06d40717d0 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.420+0000 7f06d8a35640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c66e0 con 0x7f06d40cca40 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.420+0000 7f06d8a35640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c5b60 con 0x7f06d4143c40 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.420+0000 7f06d2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d413e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50756/0 (socket says 172.21.15.64:50756) 2023-11-24T06:29:00.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 -- 172.21.15.64:0/4216864574 learned_addr learned my addr 172.21.15.64:0/4216864574 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06cbfff640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06d40cca40 0x7f06d40ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d359e640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d4143c40 0x7f06d41e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 -- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d4143c40 msgr2=0x7f06d41e8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d4143c40 0x7f06d41e8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 -- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06d40cca40 msgr2=0x7f06d40ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06d40cca40 0x7f06d40ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 -- 172.21.15.64:0/4216864574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0094d0 con 0x7f06d40717d0 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.421+0000 7f06d2d9d640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d413e370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c7f0 tx=0x7f06c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.422+0000 7f06cb7fe640 1 -- 172.21.15.64:0/4216864574 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4004220 con 0x7f06d40717d0 2023-11-24T06:29:00.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.422+0000 7f06cb7fe640 1 -- 172.21.15.64:0/4216864574 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c40043a0 con 0x7f06d40717d0 2023-11-24T06:29:00.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.422+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d40c5e40 con 0x7f06d40717d0 2023-11-24T06:29:00.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.422+0000 7f06cb7fe640 1 -- 172.21.15.64:0/4216864574 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4007860 con 0x7f06d40717d0 2023-11-24T06:29:00.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.422+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d40c6400 con 0x7f06d40717d0 2023-11-24T06:29:00.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.423+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d4067450 con 0x7f06d40717d0 2023-11-24T06:29:00.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.424+0000 7f06cb7fe640 1 -- 172.21.15.64:0/4216864574 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06c4007bd0 con 0x7f06d40717d0 2023-11-24T06:29:00.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.425+0000 7f06cb7fe640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f06b406ae70 0x7f06b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.425+0000 7f06cb7fe640 1 -- 172.21.15.64:0/4216864574 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5452+0+0 (secure 0 0 0) 0x7f06c4089540 con 0x7f06d40717d0 2023-11-24T06:29:00.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.425+0000 7f06cbfff640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f06b406ae70 0x7f06b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.426+0000 7f06cbfff640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f06b406ae70 0x7f06b406d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f06bc036000 tx=0x7f06bc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:00.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.429+0000 7f06cb7fe640 1 -- 172.21.15.64:0/4216864574 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06c4054650 con 0x7f06d40717d0 2023-11-24T06:29:00.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:00.624+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06d40ceb30 con 0x7f06d40717d0 2023-11-24T06:29:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.180+0000 7f06cb7fe640 1 -- 172.21.15.64:0/4216864574 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f06c4057c70 con 0x7f06d40717d0 2023-11-24T06:29:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f06b406ae70 msgr2=0x7f06b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f06b406ae70 0x7f06b406d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f06bc036000 tx=0x7f06bc03d040 comp rx=0 tx=0).stop 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 msgr2=0x7f06d413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d413e370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c7f0 tx=0x7f06c400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 shutdown_connections 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f06b406ae70 0x7f06b406d330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06d4143c40 0x7f06d41e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06d40cca40 0x7f06d40ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 --2- 172.21.15.64:0/4216864574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d40717d0 0x7f06d413e370 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 >> 172.21.15.64:0/4216864574 conn(0x7f06d406a800 msgr2=0x7f06d40af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 shutdown_connections 2023-11-24T06:29:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.184+0000 7f06d8a35640 1 -- 172.21.15.64:0/4216864574 wait complete. 2023-11-24T06:29:01.207 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:29:01.208 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:29:01.208 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:29:01.208 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:01.208 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:29:01.208 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:29:01.208 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:29:01.209 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:01.209 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:29:01.209 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:29:01.209 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:29:01.209 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:01.209 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:29:01.209 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:29:01.210 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:29:01.210 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:01.236 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:29:01.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:29:01.237 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:29:01.238 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:29:01.239 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:29:01.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:29:01.240 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:29:01.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:29:01.285 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:29:01.285 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:01.289 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:29:01.289 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:01.290 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:29:01.291 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:01.297 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:29:01.297 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:01.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:29:01.441 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2023-11-24T06:29:01.442 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2023-11-24T06:29:01.463 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T06:29:01.463 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T06:29:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.491+0000 7f9ebc583640 1 Processor -- start 2023-11-24T06:29:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.492+0000 7f9ebc583640 1 -- start start 2023-11-24T06:29:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebc583640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb4143ae0 0x7f9eb40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebc583640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9eb415e3b0 0x7f9eb40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebc583640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb40cfc30 con 0x7f9eb40cf7d0 2023-11-24T06:29:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebc583640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb40b1ee0 con 0x7f9eb415e3b0 2023-11-24T06:29:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebc583640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40b2780 con 0x7f9eb4143ae0 2023-11-24T06:29:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebab20640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9eb415e3b0 0x7f9eb40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebab20640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9eb415e3b0 0x7f9eb40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50794/0 (socket says 172.21.15.64:50794) 2023-11-24T06:29:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9ebab20640 1 -- 172.21.15.64:0/3456080871 learned_addr learned my addr 172.21.15.64:0/3456080871 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9eb9b1e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9eb40cf7d0 0x7f9eb415dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39114/0 (socket says 172.21.15.64:39114) 2023-11-24T06:29:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.493+0000 7f9eba31f640 1 --2- 172.21.15.64:0/3456080871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb4143ae0 0x7f9eb40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.494+0000 7f9ebab20640 1 -- 172.21.15.64:0/3456080871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb4143ae0 msgr2=0x7f9eb40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:01.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.494+0000 7f9ebab20640 1 --2- 172.21.15.64:0/3456080871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb4143ae0 0x7f9eb40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.494+0000 7f9ebab20640 1 -- 172.21.15.64:0/3456080871 >> v1:172.21.15.64:6789/0 conn(0x7f9eb40cf7d0 legacy=0x7f9eb415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.494+0000 7f9ebab20640 1 -- 172.21.15.64:0/3456080871 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb415e990 con 0x7f9eb415e3b0 2023-11-24T06:29:01.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.494+0000 7f9eb931d640 1 -- 172.21.15.64:0/3456080871 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126266360 0 0) 0x7f9eb40cfc30 con 0x7f9eb40cf7d0 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.495+0000 7f9ebab20640 1 --2- 172.21.15.64:0/3456080871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb415e3b0 0x7f9eb40702c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009e50 tx=0x7f9eac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36f1368f50dc0b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.495+0000 7f9eb931d640 1 -- 172.21.15.64:0/3456080871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac02cb00 con 0x7f9eb415e3b0 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.495+0000 7f9eb931d640 1 -- 172.21.15.64:0/3456080871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac02cc60 con 0x7f9eb415e3b0 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.495+0000 7f9eb931d640 1 -- 172.21.15.64:0/3456080871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0343f0 con 0x7f9eb415e3b0 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 -- 172.21.15.64:0/3456080871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb415e3b0 msgr2=0x7f9eb40702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 --2- 172.21.15.64:0/3456080871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb415e3b0 0x7f9eb40702c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009e50 tx=0x7f9eac02bf60 comp rx=0 tx=0).stop 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 -- 172.21.15.64:0/3456080871 shutdown_connections 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 --2- 172.21.15.64:0/3456080871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb415e3b0 0x7f9eb40702c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 --2- 172.21.15.64:0/3456080871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb4143ae0 0x7f9eb40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 -- 172.21.15.64:0/3456080871 >> 172.21.15.64:0/3456080871 conn(0x7f9eb406a800 msgr2=0x7f9eb406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:01.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 -- 172.21.15.64:0/3456080871 shutdown_connections 2023-11-24T06:29:01.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.496+0000 7f9ebc583640 1 -- 172.21.15.64:0/3456080871 wait complete. 2023-11-24T06:29:01.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.498+0000 7f9ebc583640 1 Processor -- start 2023-11-24T06:29:01.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.498+0000 7f9ebc583640 1 -- start start 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.499+0000 7f9ebc583640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cf7d0 0x7f9eb413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.499+0000 7f9ebc583640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb4143ae0 0x7f9eb413ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.499+0000 7f9ebc583640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb415e3b0 0x7f9eb413ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.499+0000 7f9ebc583640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb413d210 con 0x7f9eb4143ae0 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.499+0000 7f9ebc583640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4070f00 con 0x7f9eb415e3b0 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.499+0000 7f9ebc583640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb413cb50 con 0x7f9eb40cf7d0 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb4143ae0 0x7f9eb413ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb4143ae0 0x7f9eb413ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50810/0 (socket says 172.21.15.64:50810) 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 -- 172.21.15.64:0/4288057131 learned_addr learned my addr 172.21.15.64:0/4288057131 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:01.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9ebab20640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb415e3b0 0x7f9eb413ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eba31f640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cf7d0 0x7f9eb413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 -- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cf7d0 msgr2=0x7f9eb413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cf7d0 0x7f9eb413b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 -- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb415e3b0 msgr2=0x7f9eb413ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb415e3b0 0x7f9eb413ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.500+0000 7f9eb9b1e640 1 -- 172.21.15.64:0/4288057131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac009b30 con 0x7f9eb4143ae0 2023-11-24T06:29:01.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.501+0000 7f9eba31f640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cf7d0 0x7f9eb413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:01.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.501+0000 7f9ebab20640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb415e3b0 0x7f9eb413ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:01.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.503+0000 7f9eb9b1e640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb4143ae0 0x7f9eb413ba60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c3f0 tx=0x7f9ea400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.503+0000 7f9eaaffd640 1 -- 172.21.15.64:0/4288057131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40074e0 con 0x7f9eb4143ae0 2023-11-24T06:29:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.503+0000 7f9eaaffd640 1 -- 172.21.15.64:0/4288057131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea4007640 con 0x7f9eb4143ae0 2023-11-24T06:29:01.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.503+0000 7f9eaaffd640 1 -- 172.21.15.64:0/4288057131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4004a20 con 0x7f9eb4143ae0 2023-11-24T06:29:01.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.504+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb413ce30 con 0x7f9eb4143ae0 2023-11-24T06:29:01.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.504+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb413c180 con 0x7f9eb4143ae0 2023-11-24T06:29:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.506+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c005250 con 0x7f9eb4143ae0 2023-11-24T06:29:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.507+0000 7f9eaaffd640 1 -- 172.21.15.64:0/4288057131 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ea4004b80 con 0x7f9eb4143ae0 2023-11-24T06:29:01.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.509+0000 7f9eaaffd640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9e9806ae90 0x7f9e9806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.509+0000 7f9eaaffd640 1 -- 172.21.15.64:0/4288057131 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ea4088d90 con 0x7f9eb4143ae0 2023-11-24T06:29:01.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.510+0000 7f9eba31f640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9e9806ae90 0x7f9e9806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.511+0000 7f9eba31f640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9e9806ae90 0x7f9e9806d350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0006fd0 tx=0x7f9eb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:01.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.517+0000 7f9eaaffd640 1 -- 172.21.15.64:0/4288057131 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ea40542b0 con 0x7f9eb4143ae0 2023-11-24T06:29:01.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:01.747+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9e9c005500 con 0x7f9eb4143ae0 2023-11-24T06:29:02.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.807+0000 7f9eaaffd640 1 -- 172.21.15.64:0/4288057131 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9ea40578d0 con 0x7f9eb4143ae0 2023-11-24T06:29:02.811 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-11-24T06:29:02.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9e9806ae90 msgr2=0x7f9e9806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9e9806ae90 0x7f9e9806d350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0006fd0 tx=0x7f9eb0008040 comp rx=0 tx=0).stop 2023-11-24T06:29:02.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb4143ae0 msgr2=0x7f9eb413ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb4143ae0 0x7f9eb413ba60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c3f0 tx=0x7f9ea400c8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 shutdown_connections 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9e9806ae90 0x7f9e9806d350 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb415e3b0 0x7f9eb413ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eb4143ae0 0x7f9eb413ba60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 --2- 172.21.15.64:0/4288057131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cf7d0 0x7f9eb413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 >> 172.21.15.64:0/4288057131 conn(0x7f9eb406a800 msgr2=0x7f9eb406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.812+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 shutdown_connections 2023-11-24T06:29:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:02.813+0000 7f9ebc583640 1 -- 172.21.15.64:0/4288057131 wait complete. 2023-11-24T06:29:02.835 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:29:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.045+0000 7f467cc85640 1 Processor -- start 2023-11-24T06:29:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.045+0000 7f467cc85640 1 -- start start 2023-11-24T06:29:03.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.046+0000 7f467cc85640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4678160920 0x7f46780b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f467cc85640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46780d0a10 con 0x7f46780cf9f0 2023-11-24T06:29:03.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f467cc85640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4678161e30 con 0x7f46780cfe70 2023-11-24T06:29:03.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f467cc85640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46780d1f50 con 0x7f4678160920 2023-11-24T06:29:03.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f4676ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f46780cf9f0 0x7f4678161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33048/0 (socket says 172.21.15.64:33048) 2023-11-24T06:29:03.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f4676ffd640 1 -- 172.21.15.64:0/2006605637 learned_addr learned my addr 172.21.15.64:0/2006605637 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f46777fe640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4678160920 0x7f46780b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755022144 0 0) 0x7f46780d0a10 con 0x7f46780cf9f0 2023-11-24T06:29:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.047+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4660003520 con 0x7f46780cf9f0 2023-11-24T06:29:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3255065230 0 0) 0x7f4660003520 con 0x7f46780cf9f0 2023-11-24T06:29:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4678160920 msgr2=0x7f46780b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4678160920 0x7f46780b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 >> v1:172.21.15.162:6789/0 conn(0x7f46780cfe70 legacy=0x7f4678161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46780cf080 con 0x7f46780cf9f0 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f466c001670 con 0x7f46780cf9f0 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 >> v1:172.21.15.64:6789/0 conn(0x7f46780cf9f0 legacy=0x7f4678161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f4660003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600047c0 0x7f4660004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4678160920 0x7f4660005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780d1f50 con 0x7f46600039a0 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466000ba60 con 0x7f46600047c0 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466000bc20 con 0x7f4678160920 2023-11-24T06:29:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.048+0000 7f46777fe640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f4660003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.049+0000 7f46777fe640 1 -- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4678160920 msgr2=0x7f4660005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.049+0000 7f46777fe640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4678160920 0x7f4660005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.049+0000 7f46777fe640 1 -- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600047c0 msgr2=0x7f4660004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.049+0000 7f46777fe640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600047c0 0x7f4660004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.049+0000 7f46777fe640 1 -- 172.21.15.64:0/2006605637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660003520 con 0x7f46600039a0 2023-11-24T06:29:03.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.050+0000 7f46777fe640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f4660003dc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4668009a90 tx=0x7f46600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d4851111898240 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.050+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466802c6b0 con 0x7f46600039a0 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.050+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466802c810 con 0x7f46600039a0 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.050+0000 7f46767fc640 1 -- 172.21.15.64:0/2006605637 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466802cba0 con 0x7f46600039a0 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.051+0000 7f467cc85640 1 -- 172.21.15.64:0/2006605637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 msgr2=0x7f4660003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.051+0000 7f467cc85640 1 --2- 172.21.15.64:0/2006605637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f4660003dc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4668009a90 tx=0x7f46600057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.051+0000 7f4677fff640 1 -- 172.21.15.64:0/2006605637 reap_dead start 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.051+0000 7f467cc85640 1 -- 172.21.15.64:0/2006605637 shutdown_connections 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.051+0000 7f467cc85640 1 -- 172.21.15.64:0/2006605637 >> 172.21.15.64:0/2006605637 conn(0x7f46780d3090 msgr2=0x7f46780d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.051+0000 7f467cc85640 1 -- 172.21.15.64:0/2006605637 shutdown_connections 2023-11-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.051+0000 7f467cc85640 1 -- 172.21.15.64:0/2006605637 wait complete. 2023-11-24T06:29:03.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.053+0000 7f467cc85640 1 Processor -- start 2023-11-24T06:29:03.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.053+0000 7f467cc85640 1 -- start start 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f467cc85640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f46781f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f467cc85640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f467815c6e0 0x7f4678156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f467cc85640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46780b70e0 0x7f46780b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f467cc85640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780ca160 con 0x7f46600039a0 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f467cc85640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780b7aa0 con 0x7f467815c6e0 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f467cc85640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46781f2ce0 con 0x7f46780b70e0 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f46777fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f46781f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f46777fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f46781f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56128/0 (socket says 172.21.15.64:56128) 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.054+0000 7f46777fe640 1 -- 172.21.15.64:0/1185617874 learned_addr learned my addr 172.21.15.64:0/1185617874 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f46777fe640 1 -- 172.21.15.64:0/1185617874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46780b70e0 msgr2=0x7f46780b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f46777fe640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46780b70e0 0x7f46780b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f46777fe640 1 -- 172.21.15.64:0/1185617874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f467815c6e0 msgr2=0x7f4678156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f46777fe640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f467815c6e0 0x7f4678156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f46777fe640 1 -- 172.21.15.64:0/1185617874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46781f6900 con 0x7f46600039a0 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f46777fe640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f46781f63c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f466802c250 tx=0x7f4668004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f465ffff640 1 -- 172.21.15.64:0/1185617874 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668003810 con 0x7f46600039a0 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f465ffff640 1 -- 172.21.15.64:0/1185617874 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668003970 con 0x7f46600039a0 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f465ffff640 1 -- 172.21.15.64:0/1185617874 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668003d20 con 0x7f46600039a0 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.055+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46781f6ba0 con 0x7f46600039a0 2023-11-24T06:29:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.056+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46781f7160 con 0x7f46600039a0 2023-11-24T06:29:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.057+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46781f72f0 con 0x7f46600039a0 2023-11-24T06:29:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.057+0000 7f465ffff640 1 -- 172.21.15.64:0/1185617874 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4668040450 con 0x7f46600039a0 2023-11-24T06:29:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.058+0000 7f465ffff640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f466406ae70 0x7f466406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.058+0000 7f465ffff640 1 -- 172.21.15.64:0/1185617874 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f46680ab0d0 con 0x7f46600039a0 2023-11-24T06:29:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.059+0000 7f4676ffd640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f466406ae70 0x7f466406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.059+0000 7f4676ffd640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f466406ae70 0x7f466406d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46780c7260 tx=0x7f466c001580 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.063+0000 7f465ffff640 1 -- 172.21.15.64:0/1185617874 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4668076430 con 0x7f46600039a0 2023-11-24T06:29:03.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.329+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4678067450 con 0x7f46600039a0 2023-11-24T06:29:03.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.330+0000 7f465ffff640 1 -- 172.21.15.64:0/1185617874 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f4668079a50 con 0x7f46600039a0 2023-11-24T06:29:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:03.333 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-11-24T06:29:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAVQmBl5LPBIBAAK8ZrBVR74BsWwdMl+o3pIQ==", 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-11-24T06:29:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAWQmBlnzDUCBAATUyJ5k3zNicQ1LF6VcGuhg==", 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAVQmBlyq59MhAArlK8MP3EbOGju5GCkdv2CA==", 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAWQmBlEdAhGhAAGQy68F64p4yFOxLDRF/w8A==", 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T06:29:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAbQmBlTdDlIRAAWRp8wd1QtGOYfBwSPCGBYQ==", 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAfQmBl4JnSARAAeFR52EVl6b4P/qr1q4KJhQ==", 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAiQmBlhVMCDRAAlBHIaLz4BpzJRhgZ9XaCOQ==", 2023-11-24T06:29:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAlQmBlwjV4JxAALMfIK6BnDxZHx6/ZWfXnFw==", 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAsQmBlZ6tfNBAA0h5fM+S2ktgYd1V0Eeg94w==", 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAwQmBlz1ZdAxAABR5oSb2a2Qx6JivcAXfndg==", 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAzQmBl6V9OHBAAvRq6Ej/lF5GhuKVV73C3nA==", 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-11-24T06:29:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA2QmBlG39mJhAAMooP5segdadKlTamXvsy2w==", 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAWQmBlHzE7KRAAcsll2ZSOmVWCava8K2aOxA==", 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAXQmBla5RjBhAADHjknLo81m7W0rdy4F1K5w==", 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAWQmBlk5N8NRAA55v2xj4Lz9pKwiXNf3Jd8g==", 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T06:29:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAXQmBlcFw2ERAAm4XGpcaioQYWZBbFD3D1XA==", 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAUQmBl/5NbFBAAQJcjT/V/3G6zddzUCd9TOw==", 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mds", 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBKQmBl6A12ABAAb4NDGmyeR5J7l9TlVwATVw==", 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBKQmBl8C12ABAAPKXkNceGAf34kacE13irCg==", 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-osd", 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBKQmBlGEd2ABAA8waT8c1r6UOGoWSxzofFiQ==", 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBKQmBl0l92ABAAlmLmlw9yngqVs6yjvcbKJQ==", 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T06:29:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBKQmBlFHl2ABAAtIT2xfsfJYBRyBzY5M+Jwg==", 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBKQmBlCJN2ABAA3qL0/jef0mxdF3YEo8tlsA==", 2023-11-24T06:29:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAVQmBlFouwFBAAMw85ZoHBMGIUUyvoHbpwsA==", 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-11-24T06:29:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAVQmBloVmkCBAA6iwoOA2UZF0lcB3gEpGU8g==", 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:03.353 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f466406ae70 msgr2=0x7f466406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f466406ae70 0x7f466406d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46780c7260 tx=0x7f466c001580 comp rx=0 tx=0).stop 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 msgr2=0x7f46781f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f46781f63c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f466802c250 tx=0x7f4668004730 comp rx=0 tx=0).stop 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 shutdown_connections 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f466406ae70 0x7f466406d330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46780b70e0 0x7f46780b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f467815c6e0 0x7f4678156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 --2- 172.21.15.64:0/1185617874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46600039a0 0x7f46781f63c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 >> 172.21.15.64:0/1185617874 conn(0x7f46780d3090 msgr2=0x7f467806b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 shutdown_connections 2023-11-24T06:29:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.333+0000 7f467cc85640 1 -- 172.21.15.64:0/1185617874 wait complete. 2023-11-24T06:29:03.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T06:29:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.572+0000 7f7b35d61640 1 Processor -- start 2023-11-24T06:29:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.573+0000 7f7b35d61640 1 -- start start 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.574+0000 7f7b35d61640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 0x7f7b30166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.574+0000 7f7b35d61640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3015c5f0 con 0x7f7b3013df80 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.574+0000 7f7b35d61640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3006c470 con 0x7f7b3013e360 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.574+0000 7f7b35d61640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3006c750 con 0x7f7b300cf9a0 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.575+0000 7f7b2f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7b3013df80 0x7f7b3015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33050/0 (socket says 172.21.15.64:33050) 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.575+0000 7f7b2f7fe640 1 -- 172.21.15.64:0/3362998655 learned_addr learned my addr 172.21.15.64:0/3362998655 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.575+0000 7f7b2ffff640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 0x7f7b30166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.575+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172828354 0 0) 0x7f7b3015c5f0 con 0x7f7b3013df80 2023-11-24T06:29:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.575+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b1c003520 con 0x7f7b3013df80 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2828926338 0 0) 0x7f7b1c003520 con 0x7f7b3013df80 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 msgr2=0x7f7b30166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 0x7f7b30166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 >> v1:172.21.15.162:6789/0 conn(0x7f7b3013e360 legacy=0x7f7b300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3006ca30 con 0x7f7b3013df80 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f7b20001670 con 0x7f7b3013df80 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 >> v1:172.21.15.64:6789/0 conn(0x7f7b3013df80 legacy=0x7f7b3015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1c0039a0 0x7f7b1c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 0x7f7b1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c004d00 0x7f7b1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3006c750 con 0x7f7b1c0039a0 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c00ba60 con 0x7f7b1c004d00 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c00bc20 con 0x7f7b300cf9a0 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1c0039a0 0x7f7b1c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.576+0000 7f7b2ffff640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c004d00 0x7f7b1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b27fff640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 0x7f7b1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b2f7fe640 1 -- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 msgr2=0x7f7b1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 0x7f7b1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b2f7fe640 1 -- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c004d00 msgr2=0x7f7b1c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c004d00 0x7f7b1c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b2f7fe640 1 -- 172.21.15.64:0/3362998655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c003520 con 0x7f7b1c0039a0 2023-11-24T06:29:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b2ffff640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c004d00 0x7f7b1c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.577+0000 7f7b27fff640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b300cf9a0 0x7f7b1c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.578+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1c0039a0 0x7f7b1c003dc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c210 tx=0x7f7b1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2589a33649dbf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.578+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20005ad0 con 0x7f7b1c0039a0 2023-11-24T06:29:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.578+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b20005c30 con 0x7f7b1c0039a0 2023-11-24T06:29:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.578+0000 7f7b2effd640 1 -- 172.21.15.64:0/3362998655 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002c890 con 0x7f7b1c0039a0 2023-11-24T06:29:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.578+0000 7f7b35d61640 1 -- 172.21.15.64:0/3362998655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1c0039a0 msgr2=0x7f7b1c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.579+0000 7f7b35d61640 1 --2- 172.21.15.64:0/3362998655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1c0039a0 0x7f7b1c003dc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c210 tx=0x7f7b1c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.579+0000 7f7b2ffff640 1 -- 172.21.15.64:0/3362998655 reap_dead start 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.579+0000 7f7b35d61640 1 -- 172.21.15.64:0/3362998655 shutdown_connections 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.579+0000 7f7b35d61640 1 -- 172.21.15.64:0/3362998655 >> 172.21.15.64:0/3362998655 conn(0x7f7b3006a800 msgr2=0x7f7b3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.579+0000 7f7b35d61640 1 -- 172.21.15.64:0/3362998655 shutdown_connections 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.579+0000 7f7b35d61640 1 -- 172.21.15.64:0/3362998655 wait complete. 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.581+0000 7f7b35d61640 1 Processor -- start 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.581+0000 7f7b35d61640 1 -- start start 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b35d61640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c0039a0 0x7f7b30150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b35d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3013db20 0x7f7b3014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b35d61640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b3014c670 0x7f7b3014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b35d61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3014df40 con 0x7f7b3013db20 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b35d61640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3014c1e0 con 0x7f7b1c0039a0 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b35d61640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3014b8b0 con 0x7f7b3014c670 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b27fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3013db20 0x7f7b3014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b27fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3013db20 0x7f7b3014ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56152/0 (socket says 172.21.15.64:56152) 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b27fff640 1 -- 172.21.15.64:0/2020953591 learned_addr learned my addr 172.21.15.64:0/2020953591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b2ffff640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b3014c670 0x7f7b3014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c0039a0 0x7f7b30150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.582+0000 7f7b27fff640 1 -- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b3014c670 msgr2=0x7f7b3014f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b27fff640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b3014c670 0x7f7b3014f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b27fff640 1 -- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c0039a0 msgr2=0x7f7b30150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b27fff640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c0039a0 0x7f7b30150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b27fff640 1 -- 172.21.15.64:0/2020953591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3014bb90 con 0x7f7b3013db20 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c0039a0 0x7f7b30150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b2ffff640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b3014c670 0x7f7b3014f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b27fff640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3013db20 0x7f7b3014ecf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800d7a0 tx=0x7f7b1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b2cff9640 1 -- 172.21.15.64:0/2020953591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18017070 con 0x7f7b3013db20 2023-11-24T06:29:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b2cff9640 1 -- 172.21.15.64:0/2020953591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b18005070 con 0x7f7b3013db20 2023-11-24T06:29:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b2cff9640 1 -- 172.21.15.64:0/2020953591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18005420 con 0x7f7b3013db20 2023-11-24T06:29:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3014be70 con 0x7f7b3013db20 2023-11-24T06:29:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.583+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b301512b0 con 0x7f7b3013db20 2023-11-24T06:29:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.585+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3014c000 con 0x7f7b3013db20 2023-11-24T06:29:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.585+0000 7f7b2cff9640 1 -- 172.21.15.64:0/2020953591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b18007030 con 0x7f7b3013db20 2023-11-24T06:29:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.586+0000 7f7b2cff9640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b1006ae70 0x7f7b1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.587+0000 7f7b2cff9640 1 -- 172.21.15.64:0/2020953591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7b18088c60 con 0x7f7b3013db20 2023-11-24T06:29:03.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.587+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b1006ae70 0x7f7b1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.587+0000 7f7b2f7fe640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b1006ae70 0x7f7b1006d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7b20034000 tx=0x7f7b20009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.591+0000 7f7b2cff9640 1 -- 172.21.15.64:0/2020953591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b18054080 con 0x7f7b3013db20 2023-11-24T06:29:03.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.859+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7b300675d0 con 0x7f7b3013db20 2023-11-24T06:29:03.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.863+0000 7f7b2cff9640 1 -- 172.21.15.64:0/2020953591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7b180576a0 con 0x7f7b3013db20 2023-11-24T06:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b1006ae70 msgr2=0x7f7b1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b1006ae70 0x7f7b1006d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7b20034000 tx=0x7f7b20009030 comp rx=0 tx=0).stop 2023-11-24T06:29:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3013db20 msgr2=0x7f7b3014ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3013db20 0x7f7b3014ecf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800d7a0 tx=0x7f7b1800dc60 comp rx=0 tx=0).stop 2023-11-24T06:29:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 shutdown_connections 2023-11-24T06:29:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7b1006ae70 0x7f7b1006d330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b3014c670 0x7f7b3014f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3013db20 0x7f7b3014ecf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 --2- 172.21.15.64:0/2020953591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b1c0039a0 0x7f7b30150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 >> 172.21.15.64:0/2020953591 conn(0x7f7b3006a800 msgr2=0x7f7b3015dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 shutdown_connections 2023-11-24T06:29:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:03.867+0000 7f7b35d61640 1 -- 172.21.15.64:0/2020953591 wait complete. 2023-11-24T06:29:03.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T06:29:04.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.096+0000 7f23593a7640 1 Processor -- start 2023-11-24T06:29:04.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.096+0000 7f23593a7640 1 -- start start 2023-11-24T06:29:04.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f23593a7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f235406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f23593a7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2354071100 0x7f2354071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f23593a7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f235416a2b0 0x7f235416a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f23593a7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235406e5c0 con 0x7f2354071100 2023-11-24T06:29:04.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f23593a7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23540b36d0 con 0x7f235416a2b0 2023-11-24T06:29:04.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f23593a7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2354071aa0 con 0x7f23541642a0 2023-11-24T06:29:04.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f234bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2354071100 0x7f2354071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f234bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2354071100 0x7f2354071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56162/0 (socket says 172.21.15.64:56162) 2023-11-24T06:29:04.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.098+0000 7f234bfff640 1 -- 172.21.15.64:0/1937535839 learned_addr learned my addr 172.21.15.64:0/1937535839 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:04.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.099+0000 7f2353fff640 1 --2- 172.21.15.64:0/1937535839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f235406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.099+0000 7f2358ba6640 1 --2- 172.21.15.64:0/1937535839 >> v2:172.21.15.162:3300/0 conn(0x7f235416a2b0 0x7f235416a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.111+0000 7f234bfff640 1 -- 172.21.15.64:0/1937535839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 msgr2=0x7f235406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:04.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.111+0000 7f234bfff640 1 --2- 172.21.15.64:0/1937535839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f235406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.111+0000 7f234bfff640 1 -- 172.21.15.64:0/1937535839 >> v2:172.21.15.162:3300/0 conn(0x7f235416a2b0 msgr2=0x7f235416a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:04.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.111+0000 7f234bfff640 1 --2- 172.21.15.64:0/1937535839 >> v2:172.21.15.162:3300/0 conn(0x7f235416a2b0 0x7f235416a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.111+0000 7f234bfff640 1 -- 172.21.15.64:0/1937535839 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235416ac80 con 0x7f2354071100 2023-11-24T06:29:04.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.112+0000 7f234bfff640 1 --2- 172.21.15.64:0/1937535839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f2354071560 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f23540cb8f0 tx=0x7f233c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d705d8f825bd9d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23537fe640 1 -- 172.21.15.64:0/1937535839 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c02ce40 con 0x7f2354071100 2023-11-24T06:29:04.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23537fe640 1 -- 172.21.15.64:0/1937535839 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c030070 con 0x7f2354071100 2023-11-24T06:29:04.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23537fe640 1 -- 172.21.15.64:0/1937535839 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c036620 con 0x7f2354071100 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23593a7640 1 -- 172.21.15.64:0/1937535839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 msgr2=0x7f2354071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23593a7640 1 --2- 172.21.15.64:0/1937535839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f2354071560 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f23540cb8f0 tx=0x7f233c02c0f0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23593a7640 1 -- 172.21.15.64:0/1937535839 shutdown_connections 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23593a7640 1 --2- 172.21.15.64:0/1937535839 >> v2:172.21.15.162:3300/0 conn(0x7f235416a2b0 0x7f235416a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23593a7640 1 --2- 172.21.15.64:0/1937535839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f2354071560 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23593a7640 1 --2- 172.21.15.64:0/1937535839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f235406cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.113+0000 7f23593a7640 1 -- 172.21.15.64:0/1937535839 >> 172.21.15.64:0/1937535839 conn(0x7f23540b0b70 msgr2=0x7f23540b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.114+0000 7f23593a7640 1 -- 172.21.15.64:0/1937535839 shutdown_connections 2023-11-24T06:29:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.114+0000 7f23593a7640 1 -- 172.21.15.64:0/1937535839 wait complete. 2023-11-24T06:29:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.115+0000 7f23593a7640 1 Processor -- start 2023-11-24T06:29:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.116+0000 7f23593a7640 1 -- start start 2023-11-24T06:29:04.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f23593a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f23541553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f23593a7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f23541f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f23593a7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f235416a2b0 0x7f23541f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f23593a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23541679b0 con 0x7f2354071100 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f23593a7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2354163870 con 0x7f235416a2b0 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f23593a7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2354166d30 con 0x7f23541642a0 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f23541553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f23541553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56172/0 (socket says 172.21.15.64:56172) 2023-11-24T06:29:04.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 -- 172.21.15.64:0/3345648946 learned_addr learned my addr 172.21.15.64:0/3345648946 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f234bfff640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f23541f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 -- 172.21.15.64:0/3345648946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 msgr2=0x7f23541f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f23541f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 -- 172.21.15.64:0/3345648946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f235416a2b0 msgr2=0x7f23541f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f235416a2b0 0x7f23541f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.117+0000 7f2353fff640 1 -- 172.21.15.64:0/3345648946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c00b5d0 con 0x7f2354071100 2023-11-24T06:29:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.118+0000 7f2353fff640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f23541553d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f234400ef80 tx=0x7f2344013490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.118+0000 7f23517fa640 1 -- 172.21.15.64:0/3345648946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344015040 con 0x7f2354071100 2023-11-24T06:29:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.118+0000 7f23517fa640 1 -- 172.21.15.64:0/3345648946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234400e0d0 con 0x7f2354071100 2023-11-24T06:29:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.118+0000 7f23517fa640 1 -- 172.21.15.64:0/3345648946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234400e440 con 0x7f2354071100 2023-11-24T06:29:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.118+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2354167010 con 0x7f2354071100 2023-11-24T06:29:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.118+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23541675d0 con 0x7f2354071100 2023-11-24T06:29:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.119+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2354167c20 con 0x7f2354071100 2023-11-24T06:29:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.120+0000 7f23517fa640 1 -- 172.21.15.64:0/3345648946 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f234400c730 con 0x7f2354071100 2023-11-24T06:29:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.121+0000 7f23517fa640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f233406ae40 0x7f233406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.121+0000 7f23517fa640 1 -- 172.21.15.64:0/3345648946 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f234408d7a0 con 0x7f2354071100 2023-11-24T06:29:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.122+0000 7f234bfff640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f233406ae40 0x7f233406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.122+0000 7f234bfff640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f233406ae40 0x7f233406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f233c037000 tx=0x7f233c032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.126+0000 7f23517fa640 1 -- 172.21.15.64:0/3345648946 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2344058ba0 con 0x7f2354071100 2023-11-24T06:29:04.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.397+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f2354067450 con 0x7f2354071100 2023-11-24T06:29:04.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.400+0000 7f23517fa640 1 -- 172.21.15.64:0/3345648946 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f234405c1c0 con 0x7f2354071100 2023-11-24T06:29:04.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.403+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f233406ae40 msgr2=0x7f233406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.403+0000 7f23593a7640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f233406ae40 0x7f233406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f233c037000 tx=0x7f233c032000 comp rx=0 tx=0).stop 2023-11-24T06:29:04.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 msgr2=0x7f23541553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f23541553d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f234400ef80 tx=0x7f2344013490 comp rx=0 tx=0).stop 2023-11-24T06:29:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 shutdown_connections 2023-11-24T06:29:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f233406ae40 0x7f233406d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f235416a2b0 0x7f23541f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23541642a0 0x7f23541f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 --2- 172.21.15.64:0/3345648946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2354071100 0x7f23541553d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 >> 172.21.15.64:0/3345648946 conn(0x7f23540b0b70 msgr2=0x7f23540b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 shutdown_connections 2023-11-24T06:29:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.404+0000 7f23593a7640 1 -- 172.21.15.64:0/3345648946 wait complete. 2023-11-24T06:29:04.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T06:29:04.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.648+0000 7f01ddfa9640 1 Processor -- start 2023-11-24T06:29:04.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.648+0000 7f01ddfa9640 1 -- start start 2023-11-24T06:29:04.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.649+0000 7f01ddfa9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01d816da70 0x7f01d816de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.649+0000 7f01ddfa9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01d80ce620 0x7f01d8070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.649+0000 7f01ddfa9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d8071e00 con 0x7f01d80702c0 2023-11-24T06:29:04.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.650+0000 7f01ddfa9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80cea00 con 0x7f01d80ce620 2023-11-24T06:29:04.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.650+0000 7f01ddfa9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d816e390 con 0x7f01d816da70 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.650+0000 7f01d7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01d80ce620 0x7f01d8070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.650+0000 7f01d7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01d80ce620 0x7f01d8070750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56176/0 (socket says 172.21.15.64:56176) 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.650+0000 7f01d7fff640 1 -- 172.21.15.64:0/1501795194 learned_addr learned my addr 172.21.15.64:0/1501795194 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.651+0000 7f01d67fc640 1 -- 172.21.15.64:0/1501795194 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750860812 0 0) 0x7f01d8071e00 con 0x7f01d80702c0 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.650+0000 7f01d77fe640 1 --2- 172.21.15.64:0/1501795194 >> v2:172.21.15.162:3300/0 conn(0x7f01d816da70 0x7f01d816de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.717+0000 7f01d67fc640 1 -- 172.21.15.64:0/1501795194 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01c0005730 con 0x7f01d80702c0 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.718+0000 7f01d7fff640 1 -- 172.21.15.64:0/1501795194 >> v2:172.21.15.162:3300/0 conn(0x7f01d816da70 msgr2=0x7f01d816de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.718+0000 7f01d7fff640 1 --2- 172.21.15.64:0/1501795194 >> v2:172.21.15.162:3300/0 conn(0x7f01d816da70 0x7f01d816de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.718+0000 7f01d7fff640 1 -- 172.21.15.64:0/1501795194 >> v1:172.21.15.162:6789/0 conn(0x7f01d80702c0 legacy=0x7f01d80cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.718+0000 7f01d7fff640 1 -- 172.21.15.64:0/1501795194 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8070d60 con 0x7f01d80ce620 2023-11-24T06:29:04.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.718+0000 7f01d77fe640 1 --2- 172.21.15.64:0/1501795194 >> v2:172.21.15.162:3300/0 conn(0x7f01d816da70 0x7f01d816de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:04.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.722+0000 7f01d7fff640 1 --2- 172.21.15.64:0/1501795194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d8070750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f01d8072470 tx=0x7f01d002c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428b7daa9da1dda4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.722+0000 7f01d67fc640 1 -- 172.21.15.64:0/1501795194 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0034640 con 0x7f01d80ce620 2023-11-24T06:29:04.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.722+0000 7f01d67fc640 1 -- 172.21.15.64:0/1501795194 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00347a0 con 0x7f01d80ce620 2023-11-24T06:29:04.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.722+0000 7f01d67fc640 1 -- 172.21.15.64:0/1501795194 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0034b30 con 0x7f01d80ce620 2023-11-24T06:29:04.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.723+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1501795194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 msgr2=0x7f01d8070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.724+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1501795194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d8070750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f01d8072470 tx=0x7f01d002c290 comp rx=0 tx=0).stop 2023-11-24T06:29:04.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.724+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1501795194 shutdown_connections 2023-11-24T06:29:04.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.724+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1501795194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d8070750 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.724+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1501795194 >> v2:172.21.15.162:3300/0 conn(0x7f01d816da70 0x7f01d816de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.724+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1501795194 >> 172.21.15.64:0/1501795194 conn(0x7f01d80b0be0 msgr2=0x7f01d80b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:04.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.724+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1501795194 shutdown_connections 2023-11-24T06:29:04.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.724+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1501795194 wait complete. 2023-11-24T06:29:04.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.728+0000 7f01ddfa9640 1 Processor -- start 2023-11-24T06:29:04.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.729+0000 7f01ddfa9640 1 -- start start 2023-11-24T06:29:04.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.731+0000 7f01ddfa9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01d80702c0 0x7f01d8089a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.731+0000 7f01ddfa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d80aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.731+0000 7f01ddfa9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01d816da70 0x7f01d80a8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.731+0000 7f01ddfa9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8089f70 con 0x7f01d80ce620 2023-11-24T06:29:04.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.731+0000 7f01ddfa9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80a90d0 con 0x7f01d80702c0 2023-11-24T06:29:04.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.731+0000 7f01ddfa9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80a93b0 con 0x7f01d816da70 2023-11-24T06:29:04.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.732+0000 7f01d6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d80aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.732+0000 7f01d77fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01d80702c0 0x7f01d8089a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.732+0000 7f01d77fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01d80702c0 0x7f01d8089a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:54042/0 (socket says 172.21.15.64:54042) 2023-11-24T06:29:04.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.732+0000 7f01d77fe640 1 -- 172.21.15.64:0/1980219985 learned_addr learned my addr 172.21.15.64:0/1980219985 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:04.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.732+0000 7f01d7fff640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01d816da70 0x7f01d80a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.733+0000 7f01d77fe640 1 -- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01d816da70 msgr2=0x7f01d80a8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.733+0000 7f01d77fe640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01d816da70 0x7f01d80a8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.733+0000 7f01d77fe640 1 -- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 msgr2=0x7f01d80aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.733+0000 7f01d77fe640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d80aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.733+0000 7f01d77fe640 1 -- 172.21.15.64:0/1980219985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0008b20 con 0x7f01d80702c0 2023-11-24T06:29:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.733+0000 7f01d6ffd640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d80aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.734+0000 7f01d77fe640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01d80702c0 0x7f01d8089a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f01c80079d0 tx=0x7f01c800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.734+0000 7f01bffff640 1 -- 172.21.15.64:0/1980219985 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8018070 con 0x7f01d80702c0 2023-11-24T06:29:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.734+0000 7f01bffff640 1 -- 172.21.15.64:0/1980219985 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8004020 con 0x7f01d80702c0 2023-11-24T06:29:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.734+0000 7f01bffff640 1 -- 172.21.15.64:0/1980219985 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8004390 con 0x7f01d80702c0 2023-11-24T06:29:04.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.734+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d80a9690 con 0x7f01d80702c0 2023-11-24T06:29:04.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.735+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d80a9c50 con 0x7f01d80702c0 2023-11-24T06:29:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.737+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d80b1cd0 con 0x7f01d80702c0 2023-11-24T06:29:04.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.739+0000 7f01bffff640 1 -- 172.21.15.64:0/1980219985 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01c8003610 con 0x7f01d80702c0 2023-11-24T06:29:04.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.740+0000 7f01bffff640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01c406ae40 0x7f01c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.741+0000 7f01bffff640 1 -- 172.21.15.64:0/1980219985 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f01c8091610 con 0x7f01d80702c0 2023-11-24T06:29:04.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.741+0000 7f01d6ffd640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01c406ae40 0x7f01c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.742+0000 7f01d6ffd640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01c406ae40 0x7f01c406d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f01cc007fd0 tx=0x7f01cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:04.753+0000 7f01bffff640 1 -- 172.21.15.64:0/1980219985 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01c805cac0 con 0x7f01d80702c0 2023-11-24T06:29:05.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.021+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f01d80675d0 con 0x7f01d80702c0 2023-11-24T06:29:05.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.034+0000 7f01bffff640 1 -- 172.21.15.64:0/1980219985 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f01c802a070 con 0x7f01d80702c0 2023-11-24T06:29:05.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01c406ae40 msgr2=0x7f01c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01c406ae40 0x7f01c406d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f01cc007fd0 tx=0x7f01cc009040 comp rx=0 tx=0).stop 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01d80702c0 msgr2=0x7f01d8089a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01d80702c0 0x7f01d8089a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f01c80079d0 tx=0x7f01c800a430 comp rx=0 tx=0).stop 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 shutdown_connections 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f01c406ae40 0x7f01c406d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01d816da70 0x7f01d80a8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d80ce620 0x7f01d80aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 --2- 172.21.15.64:0/1980219985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01d80702c0 0x7f01d8089a30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.036+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 >> 172.21.15.64:0/1980219985 conn(0x7f01d80b0be0 msgr2=0x7f01d80b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.037+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 shutdown_connections 2023-11-24T06:29:05.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.037+0000 7f01ddfa9640 1 -- 172.21.15.64:0/1980219985 wait complete. 2023-11-24T06:29:05.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T06:29:05.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.284+0000 7f07a54dc640 1 Processor -- start 2023-11-24T06:29:05.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.285+0000 7f07a54dc640 1 -- start start 2023-11-24T06:29:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.286+0000 7f07a54dc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07a0160920 0x7f07a00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.286+0000 7f07a54dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07a00cf9f0 0x7f07a00cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.286+0000 7f07a54dc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00d0a10 con 0x7f07a0161010 2023-11-24T06:29:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.286+0000 7f07a54dc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00d03e0 con 0x7f07a00cf9f0 2023-11-24T06:29:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.286+0000 7f07a54dc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00d1f50 con 0x7f07a0160920 2023-11-24T06:29:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f0797fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07a00cf9f0 0x7f07a00cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f0797fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07a00cf9f0 0x7f07a00cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56192/0 (socket says 172.21.15.64:56192) 2023-11-24T06:29:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f0797fff640 1 -- 172.21.15.64:0/2523041686 learned_addr learned my addr 172.21.15.64:0/2523041686 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f079ffff640 1 --2- 172.21.15.64:0/2523041686 >> v2:172.21.15.162:3300/0 conn(0x7f07a0160920 0x7f07a00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f0797fff640 1 -- 172.21.15.64:0/2523041686 >> v2:172.21.15.162:3300/0 conn(0x7f07a0160920 msgr2=0x7f07a00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f0797fff640 1 --2- 172.21.15.64:0/2523041686 >> v2:172.21.15.162:3300/0 conn(0x7f07a0160920 0x7f07a00b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f0797fff640 1 -- 172.21.15.64:0/2523041686 >> v1:172.21.15.162:6789/0 conn(0x7f07a0161010 legacy=0x7f07a0161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.287+0000 7f0797fff640 1 -- 172.21.15.64:0/2523041686 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00cf080 con 0x7f07a00cf9f0 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.288+0000 7f0797fff640 1 --2- 172.21.15.64:0/2523041686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a00cf9f0 0x7f07a00cfea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f07a008c060 tx=0x7f078802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3560358da9647df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f079f7fe640 1 -- 172.21.15.64:0/2523041686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788031870 con 0x7f07a00cf9f0 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f079f7fe640 1 -- 172.21.15.64:0/2523041686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07880319d0 con 0x7f07a00cf9f0 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f079f7fe640 1 -- 172.21.15.64:0/2523041686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788031d60 con 0x7f07a00cf9f0 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f07a54dc640 1 -- 172.21.15.64:0/2523041686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a00cf9f0 msgr2=0x7f07a00cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f07a54dc640 1 --2- 172.21.15.64:0/2523041686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a00cf9f0 0x7f07a00cfea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f07a008c060 tx=0x7f078802b910 comp rx=0 tx=0).stop 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f07a54dc640 1 -- 172.21.15.64:0/2523041686 shutdown_connections 2023-11-24T06:29:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f07a54dc640 1 --2- 172.21.15.64:0/2523041686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a00cf9f0 0x7f07a00cfea0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f07a54dc640 1 --2- 172.21.15.64:0/2523041686 >> v2:172.21.15.162:3300/0 conn(0x7f07a0160920 0x7f07a00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.289+0000 7f07a54dc640 1 -- 172.21.15.64:0/2523041686 >> 172.21.15.64:0/2523041686 conn(0x7f07a00d3090 msgr2=0x7f07a00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.290+0000 7f07a54dc640 1 -- 172.21.15.64:0/2523041686 shutdown_connections 2023-11-24T06:29:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.290+0000 7f07a54dc640 1 -- 172.21.15.64:0/2523041686 wait complete. 2023-11-24T06:29:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.291+0000 7f07a54dc640 1 Processor -- start 2023-11-24T06:29:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.291+0000 7f07a54dc640 1 -- start start 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.292+0000 7f07a54dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a00cf9f0 0x7f07a01f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.292+0000 7f07a54dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a0160920 0x7f07a00ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.292+0000 7f07a54dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0161010 0x7f07a00cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a54dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00cc610 con 0x7f07a0161010 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a54dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00cba00 con 0x7f07a00cf9f0 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a54dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00cbce0 con 0x7f07a0160920 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0161010 0x7f07a00cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0161010 0x7f07a00cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56200/0 (socket says 172.21.15.64:56200) 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 -- 172.21.15.64:0/1706635085 learned_addr learned my addr 172.21.15.64:0/1706635085 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f079ffff640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a00cf9f0 0x7f07a01f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 -- 172.21.15.64:0/1706635085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a0160920 msgr2=0x7f07a00ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a0160920 0x7f07a00ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 -- 172.21.15.64:0/1706635085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a00cf9f0 msgr2=0x7f07a01f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a00cf9f0 0x7f07a01f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.293+0000 7f07a4cdb640 1 -- 172.21.15.64:0/1706635085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880094d0 con 0x7f07a0161010 2023-11-24T06:29:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.294+0000 7f07a4cdb640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0161010 0x7f07a00cb1b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f079800c780 tx=0x7f079800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.294+0000 7f079d7fa640 1 -- 172.21.15.64:0/1706635085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798004b60 con 0x7f07a0161010 2023-11-24T06:29:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.294+0000 7f079d7fa640 1 -- 172.21.15.64:0/1706635085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079800ce90 con 0x7f07a0161010 2023-11-24T06:29:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.294+0000 7f079d7fa640 1 -- 172.21.15.64:0/1706635085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800a850 con 0x7f07a0161010 2023-11-24T06:29:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.294+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a00cbfc0 con 0x7f07a0161010 2023-11-24T06:29:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.294+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a00cb6f0 con 0x7f07a0161010 2023-11-24T06:29:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.295+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a00cc780 con 0x7f07a0161010 2023-11-24T06:29:05.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.296+0000 7f079d7fa640 1 -- 172.21.15.64:0/1706635085 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f079800a9b0 con 0x7f07a0161010 2023-11-24T06:29:05.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.297+0000 7f079d7fa640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f078006ae40 0x7f078006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.297+0000 7f079d7fa640 1 -- 172.21.15.64:0/1706635085 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0798089490 con 0x7f07a0161010 2023-11-24T06:29:05.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.297+0000 7f079ffff640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f078006ae40 0x7f078006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.298+0000 7f079ffff640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f078006ae40 0x7f078006d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.301+0000 7f079d7fa640 1 -- 172.21.15.64:0/1706635085 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0798054880 con 0x7f07a0161010 2023-11-24T06:29:05.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.567+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f07a0067450 con 0x7f07a0161010 2023-11-24T06:29:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.571+0000 7f079d7fa640 1 -- 172.21.15.64:0/1706635085 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0798057ea0 con 0x7f07a0161010 2023-11-24T06:29:05.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f078006ae40 msgr2=0x7f078006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f078006ae40 0x7f078006d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790008040 comp rx=0 tx=0).stop 2023-11-24T06:29:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0161010 msgr2=0x7f07a00cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0161010 0x7f07a00cb1b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f079800c780 tx=0x7f079800cc40 comp rx=0 tx=0).stop 2023-11-24T06:29:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 shutdown_connections 2023-11-24T06:29:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f078006ae40 0x7f078006d300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0161010 0x7f07a00cb1b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a0160920 0x7f07a00ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 --2- 172.21.15.64:0/1706635085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a00cf9f0 0x7f07a01f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 >> 172.21.15.64:0/1706635085 conn(0x7f07a00d3090 msgr2=0x7f07a01617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 shutdown_connections 2023-11-24T06:29:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.574+0000 7f07a54dc640 1 -- 172.21.15.64:0/1706635085 wait complete. 2023-11-24T06:29:05.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T06:29:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.815+0000 7fdcf1fcf640 1 Processor -- start 2023-11-24T06:29:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.815+0000 7fdcf1fcf640 1 -- start start 2023-11-24T06:29:05.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.816+0000 7fdcf1fcf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdcec0cc500 0x7fdcec0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.816+0000 7fdcf1fcf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec06d160 0x7fdcec06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.816+0000 7fdcf1fcf640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec070e50 con 0x7fdcec143c40 2023-11-24T06:29:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.816+0000 7fdcf1fcf640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcec0723c0 con 0x7fdcec0cc500 2023-11-24T06:29:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.816+0000 7fdcf1fcf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0cced0 con 0x7fdcec06d160 2023-11-24T06:29:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.817+0000 7fdce3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdcec0cc500 0x7fdcec0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.817+0000 7fdce3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdcec0cc500 0x7fdcec0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56214/0 (socket says 172.21.15.64:56214) 2023-11-24T06:29:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.817+0000 7fdce3fff640 1 -- 172.21.15.64:0/2854887570 learned_addr learned my addr 172.21.15.64:0/2854887570 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.817+0000 7fdce3fff640 1 -- 172.21.15.64:0/2854887570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec06d160 msgr2=0x7fdcec06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.817+0000 7fdce3fff640 1 --2- 172.21.15.64:0/2854887570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec06d160 0x7fdcec06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.817+0000 7fdce3fff640 1 -- 172.21.15.64:0/2854887570 >> v1:172.21.15.162:6789/0 conn(0x7fdcec143c40 legacy=0x7fdcec071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.818+0000 7fdce3fff640 1 -- 172.21.15.64:0/2854887570 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec144020 con 0x7fdcec0cc500 2023-11-24T06:29:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.819+0000 7fdce3fff640 1 --2- 172.21.15.64:0/2854887570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0cc500 0x7fdcec0cc990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009870 tx=0x7fdcdc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4060f1d03a9be2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.819+0000 7fdce37fe640 1 -- 172.21.15.64:0/2854887570 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc031640 con 0x7fdcec0cc500 2023-11-24T06:29:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.819+0000 7fdce37fe640 1 -- 172.21.15.64:0/2854887570 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0317a0 con 0x7fdcec0cc500 2023-11-24T06:29:05.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.819+0000 7fdce37fe640 1 -- 172.21.15.64:0/2854887570 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc031b30 con 0x7fdcec0cc500 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/2854887570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0cc500 msgr2=0x7fdcec0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/2854887570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0cc500 0x7fdcec0cc990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009870 tx=0x7fdcdc02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/2854887570 shutdown_connections 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/2854887570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec06d160 0x7fdcec06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/2854887570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0cc500 0x7fdcec0cc990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/2854887570 >> 172.21.15.64:0/2854887570 conn(0x7fdcec06a800 msgr2=0x7fdcec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/2854887570 shutdown_connections 2023-11-24T06:29:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.820+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/2854887570 wait complete. 2023-11-24T06:29:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.822+0000 7fdcf1fcf640 1 Processor -- start 2023-11-24T06:29:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.822+0000 7fdcf1fcf640 1 -- start start 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf1fcf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec06d160 0x7fdcec13e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf1fcf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcec0cc500 0x7fdcec13aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf1fcf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec143c40 0x7fdcec13b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf1fcf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec1f7190 con 0x7fdcec06d160 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf1fcf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec13b550 con 0x7fdcec0cc500 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf1fcf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec1f75b0 con 0x7fdcec143c40 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf0fcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec06d160 0x7fdcec13e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf0fcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec06d160 0x7fdcec13e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56222/0 (socket says 172.21.15.64:56222) 2023-11-24T06:29:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdcf0fcd640 1 -- 172.21.15.64:0/4066538409 learned_addr learned my addr 172.21.15.64:0/4066538409 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:05.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.823+0000 7fdce3fff640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcec0cc500 0x7fdcec13aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdcf0fcd640 1 -- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec143c40 msgr2=0x7fdcec13b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdcf0fcd640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec143c40 0x7fdcec13b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdcf0fcd640 1 -- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcec0cc500 msgr2=0x7fdcec13aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdcf0fcd640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcec0cc500 0x7fdcec13aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdcf0fcd640 1 -- 172.21.15.64:0/4066538409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0094d0 con 0x7fdcec06d160 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdce3fff640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcec0cc500 0x7fdcec13aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdcf0fcd640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec06d160 0x7fdcec13e770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c740 tx=0x7fdce800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdce17fa640 1 -- 172.21.15.64:0/4066538409 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8003b90 con 0x7fdcec06d160 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdce17fa640 1 -- 172.21.15.64:0/4066538409 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce8003cf0 con 0x7fdcec06d160 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdce17fa640 1 -- 172.21.15.64:0/4066538409 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800aa90 con 0x7fdcec06d160 2023-11-24T06:29:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.824+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec13c850 con 0x7fdcec06d160 2023-11-24T06:29:05.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.825+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec1f67f0 con 0x7fdcec06d160 2023-11-24T06:29:05.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.826+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec067450 con 0x7fdcec06d160 2023-11-24T06:29:05.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.827+0000 7fdce17fa640 1 -- 172.21.15.64:0/4066538409 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdce800abf0 con 0x7fdcec06d160 2023-11-24T06:29:05.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.828+0000 7fdce17fa640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcd806ae40 0x7fdcd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.828+0000 7fdce17fa640 1 -- 172.21.15.64:0/4066538409 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdce8089010 con 0x7fdcec06d160 2023-11-24T06:29:05.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.829+0000 7fdce3fff640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcd806ae40 0x7fdcd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.829+0000 7fdce3fff640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcd806ae40 0x7fdcd806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc036000 tx=0x7fdcdc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:05.833+0000 7fdce17fa640 1 -- 172.21.15.64:0/4066538409 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdce80544a0 con 0x7fdcec06d160 2023-11-24T06:29:06.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.098+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fdcec070840 con 0x7fdcec06d160 2023-11-24T06:29:06.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.102+0000 7fdce17fa640 1 -- 172.21.15.64:0/4066538409 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdce8003e60 con 0x7fdcec06d160 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcd806ae40 msgr2=0x7fdcd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcd806ae40 0x7fdcd806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc036000 tx=0x7fdcdc03c040 comp rx=0 tx=0).stop 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec06d160 msgr2=0x7fdcec13e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec06d160 0x7fdcec13e770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c740 tx=0x7fdce800cc00 comp rx=0 tx=0).stop 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 shutdown_connections 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcd806ae40 0x7fdcd806d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcec143c40 0x7fdcec13b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcec0cc500 0x7fdcec13aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 --2- 172.21.15.64:0/4066538409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec06d160 0x7fdcec13e770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 >> 172.21.15.64:0/4066538409 conn(0x7fdcec06a800 msgr2=0x7fdcec140990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:06.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 shutdown_connections 2023-11-24T06:29:06.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.105+0000 7fdcf1fcf640 1 -- 172.21.15.64:0/4066538409 wait complete. 2023-11-24T06:29:06.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T06:29:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.346+0000 7fd3458d1640 1 Processor -- start 2023-11-24T06:29:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.346+0000 7fd3458d1640 1 -- start start 2023-11-24T06:29:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.347+0000 7fd3458d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34006c580 0x7fd3400cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.347+0000 7fd3458d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd3400cf560 0x7fd3400b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.347+0000 7fd3458d1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400cfe30 con 0x7fd34015d560 2023-11-24T06:29:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.347+0000 7fd3458d1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400d0110 con 0x7fd3400cf560 2023-11-24T06:29:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.347+0000 7fd3458d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400d03f0 con 0x7fd34006c580 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd3448cf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd34015d560 0x7fd34006be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33060/0 (socket says 172.21.15.64:33060) 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd3448cf640 1 -- 172.21.15.64:0/4046584800 learned_addr learned my addr 172.21.15.64:0/4046584800 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd3450d0640 1 --2- 172.21.15.64:0/4046584800 >> v2:172.21.15.64:3300/0 conn(0x7fd3400cf560 0x7fd3400b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd33ffff640 1 --2- 172.21.15.64:0/4046584800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34006c580 0x7fd3400cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd33f7fe640 1 -- 172.21.15.64:0/4046584800 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232641450 0 0) 0x7fd3400cfe30 con 0x7fd34015d560 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd3450d0640 1 -- 172.21.15.64:0/4046584800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34006c580 msgr2=0x7fd3400cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd3450d0640 1 --2- 172.21.15.64:0/4046584800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34006c580 0x7fd3400cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd3450d0640 1 -- 172.21.15.64:0/4046584800 >> v1:172.21.15.64:6789/0 conn(0x7fd34015d560 legacy=0x7fd34006be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.348+0000 7fd3450d0640 1 -- 172.21.15.64:0/4046584800 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400d06d0 con 0x7fd3400cf560 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.349+0000 7fd3450d0640 1 --2- 172.21.15.64:0/4046584800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3400cf560 0x7fd3400b06c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd3380097a0 tx=0x7fd33802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7bd8165b61c1f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd33f7fe640 1 -- 172.21.15.64:0/4046584800 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33802d040 con 0x7fd3400cf560 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd33f7fe640 1 -- 172.21.15.64:0/4046584800 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd338031810 con 0x7fd3400cf560 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd33f7fe640 1 -- 172.21.15.64:0/4046584800 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338031ba0 con 0x7fd3400cf560 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd3458d1640 1 -- 172.21.15.64:0/4046584800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3400cf560 msgr2=0x7fd3400b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd3458d1640 1 --2- 172.21.15.64:0/4046584800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3400cf560 0x7fd3400b06c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd3380097a0 tx=0x7fd33802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd3458d1640 1 -- 172.21.15.64:0/4046584800 shutdown_connections 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd3458d1640 1 --2- 172.21.15.64:0/4046584800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3400cf560 0x7fd3400b06c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd3458d1640 1 --2- 172.21.15.64:0/4046584800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34006c580 0x7fd3400cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.350+0000 7fd3458d1640 1 -- 172.21.15.64:0/4046584800 >> 172.21.15.64:0/4046584800 conn(0x7fd340099a60 msgr2=0x7fd340099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.351+0000 7fd3458d1640 1 -- 172.21.15.64:0/4046584800 shutdown_connections 2023-11-24T06:29:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.351+0000 7fd3458d1640 1 -- 172.21.15.64:0/4046584800 wait complete. 2023-11-24T06:29:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.352+0000 7fd3458d1640 1 Processor -- start 2023-11-24T06:29:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.352+0000 7fd3458d1640 1 -- start start 2023-11-24T06:29:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.353+0000 7fd3458d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd34006c580 0x7fd3400c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.353+0000 7fd3458d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400cf560 0x7fd3401f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.353+0000 7fd3458d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34015d560 0x7fd3401f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.353+0000 7fd3458d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401f2920 con 0x7fd34006c580 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.353+0000 7fd3458d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400c6d00 con 0x7fd3400cf560 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.353+0000 7fd3458d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400c7f50 con 0x7fd34015d560 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd34006c580 0x7fd3400c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd34006c580 0x7fd3400c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56246/0 (socket says 172.21.15.64:56246) 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 -- 172.21.15.64:0/3373917591 learned_addr learned my addr 172.21.15.64:0/3373917591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 -- 172.21.15.64:0/3373917591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34015d560 msgr2=0x7fd3401f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34015d560 0x7fd3401f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 -- 172.21.15.64:0/3373917591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400cf560 msgr2=0x7fd3401f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400cf560 0x7fd3401f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 -- 172.21.15.64:0/3373917591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd338009400 con 0x7fd34006c580 2023-11-24T06:29:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.354+0000 7fd3448cf640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd34006c580 0x7fd3400c4b80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c7c0 tx=0x7fd33400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.355+0000 7fd33d7fa640 1 -- 172.21.15.64:0/3373917591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400a640 con 0x7fd34006c580 2023-11-24T06:29:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.355+0000 7fd33d7fa640 1 -- 172.21.15.64:0/3373917591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33400a7c0 con 0x7fd34006c580 2023-11-24T06:29:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.355+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3400c8230 con 0x7fd34006c580 2023-11-24T06:29:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.355+0000 7fd33d7fa640 1 -- 172.21.15.64:0/3373917591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400ab70 con 0x7fd34006c580 2023-11-24T06:29:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.355+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3400c6fe0 con 0x7fd34006c580 2023-11-24T06:29:06.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.356+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3400c8750 con 0x7fd34006c580 2023-11-24T06:29:06.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.357+0000 7fd33d7fa640 1 -- 172.21.15.64:0/3373917591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd33400acd0 con 0x7fd34006c580 2023-11-24T06:29:06.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.358+0000 7fd33d7fa640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd32406ae40 0x7fd32406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.358+0000 7fd33d7fa640 1 -- 172.21.15.64:0/3373917591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd334088aa0 con 0x7fd34006c580 2023-11-24T06:29:06.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.358+0000 7fd33ffff640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd32406ae40 0x7fd32406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.359+0000 7fd33ffff640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd32406ae40 0x7fd32406d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd340158630 tx=0x7fd32c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.362+0000 7fd33d7fa640 1 -- 172.21.15.64:0/3373917591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3400c8750 con 0x7fd34006c580 2023-11-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.630+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd3400c8750 con 0x7fd34006c580 2023-11-24T06:29:06.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.634+0000 7fd33d7fa640 1 -- 172.21.15.64:0/3373917591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd334018020 con 0x7fd34006c580 2023-11-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd32406ae40 msgr2=0x7fd32406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd32406ae40 0x7fd32406d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd340158630 tx=0x7fd32c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd34006c580 msgr2=0x7fd3400c4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd34006c580 0x7fd3400c4b80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c7c0 tx=0x7fd33400cc80 comp rx=0 tx=0).stop 2023-11-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 shutdown_connections 2023-11-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd32406ae40 0x7fd32406d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd34015d560 0x7fd3401f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400cf560 0x7fd3401f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 --2- 172.21.15.64:0/3373917591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd34006c580 0x7fd3400c4b80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 >> 172.21.15.64:0/3373917591 conn(0x7fd340099a60 msgr2=0x7fd34015c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.638+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 shutdown_connections 2023-11-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.639+0000 7fd3458d1640 1 -- 172.21.15.64:0/3373917591 wait complete. 2023-11-24T06:29:06.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.883+0000 7fd104d11640 1 Processor -- start 2023-11-24T06:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.884+0000 7fd104d11640 1 -- start start 2023-11-24T06:29:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.885+0000 7fd104d11640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd10015d3c0 0x7fd1000d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.885+0000 7fd104d11640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.885+0000 7fd104d11640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1000b2c70 con 0x7fd1000cfa40 2023-11-24T06:29:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.885+0000 7fd104d11640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1000d03f0 con 0x7fd10015d3c0 2023-11-24T06:29:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.885+0000 7fd104d11640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000cf6d0 con 0x7fd1000b4110 2023-11-24T06:29:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0ff7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd10015d3c0 0x7fd1000d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0ff7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd10015d3c0 0x7fd1000d1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56248/0 (socket says 172.21.15.64:56248) 2023-11-24T06:29:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0ff7fe640 1 -- 172.21.15.64:0/2363412859 learned_addr learned my addr 172.21.15.64:0/2363412859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0fffff640 1 --2- 172.21.15.64:0/2363412859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0ff7fe640 1 -- 172.21.15.64:0/2363412859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 msgr2=0x7fd1000cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0ff7fe640 1 --2- 172.21.15.64:0/2363412859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0ff7fe640 1 -- 172.21.15.64:0/2363412859 >> v1:172.21.15.162:6789/0 conn(0x7fd1000cfa40 legacy=0x7fd1000cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.886+0000 7fd0ff7fe640 1 -- 172.21.15.64:0/2363412859 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000cf3f0 con 0x7fd10015d3c0 2023-11-24T06:29:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.887+0000 7fd0ff7fe640 1 --2- 172.21.15.64:0/2363412859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd10015d3c0 0x7fd1000d1f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009870 tx=0x7fd0f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b25ecd361296c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.888+0000 7fd0feffd640 1 -- 172.21.15.64:0/2363412859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0031640 con 0x7fd10015d3c0 2023-11-24T06:29:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.888+0000 7fd0feffd640 1 -- 172.21.15.64:0/2363412859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f00317a0 con 0x7fd10015d3c0 2023-11-24T06:29:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.888+0000 7fd0feffd640 1 -- 172.21.15.64:0/2363412859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0031b30 con 0x7fd10015d3c0 2023-11-24T06:29:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.888+0000 7fd104d11640 1 -- 172.21.15.64:0/2363412859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd10015d3c0 msgr2=0x7fd1000d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.888+0000 7fd104d11640 1 --2- 172.21.15.64:0/2363412859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd10015d3c0 0x7fd1000d1f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009870 tx=0x7fd0f002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.889+0000 7fd104d11640 1 -- 172.21.15.64:0/2363412859 shutdown_connections 2023-11-24T06:29:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.889+0000 7fd104d11640 1 --2- 172.21.15.64:0/2363412859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.889+0000 7fd104d11640 1 --2- 172.21.15.64:0/2363412859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd10015d3c0 0x7fd1000d1f20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.889+0000 7fd104d11640 1 -- 172.21.15.64:0/2363412859 >> 172.21.15.64:0/2363412859 conn(0x7fd1000d3050 msgr2=0x7fd1000d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.889+0000 7fd104d11640 1 -- 172.21.15.64:0/2363412859 shutdown_connections 2023-11-24T06:29:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.889+0000 7fd104d11640 1 -- 172.21.15.64:0/2363412859 wait complete. 2023-11-24T06:29:06.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.890+0000 7fd104d11640 1 Processor -- start 2023-11-24T06:29:06.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.891+0000 7fd104d11640 1 -- start start 2023-11-24T06:29:06.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.891+0000 7fd104d11640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.891+0000 7fd104d11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1000cfa40 0x7fd1000c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.891+0000 7fd104d11640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd10015d3c0 0x7fd10006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.891+0000 7fd104d11640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000c8b90 con 0x7fd1000cfa40 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.891+0000 7fd104d11640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000cc960 con 0x7fd10015d3c0 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.891+0000 7fd104d11640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000cbd50 con 0x7fd1000b4110 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1000cfa40 0x7fd1000c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1000cfa40 0x7fd1000c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56252/0 (socket says 172.21.15.64:56252) 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 -- 172.21.15.64:0/1010390637 learned_addr learned my addr 172.21.15.64:0/1010390637 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0ff7fe640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0fffff640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd10015d3c0 0x7fd10006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 -- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 msgr2=0x7fd1000cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 -- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd10015d3c0 msgr2=0x7fd10006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd10015d3c0 0x7fd10006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.892+0000 7fd0f7fff640 1 -- 172.21.15.64:0/1010390637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00094d0 con 0x7fd1000cfa40 2023-11-24T06:29:06.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.893+0000 7fd0f7fff640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1000cfa40 0x7fd1000c8650 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800c450 tx=0x7fd0e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.893+0000 7fd0fcff9640 1 -- 172.21.15.64:0/1010390637 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80052d0 con 0x7fd1000cfa40 2023-11-24T06:29:06.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.893+0000 7fd0fcff9640 1 -- 172.21.15.64:0/1010390637 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e8005430 con 0x7fd1000cfa40 2023-11-24T06:29:06.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.893+0000 7fd0fcff9640 1 -- 172.21.15.64:0/1010390637 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8012870 con 0x7fd1000cfa40 2023-11-24T06:29:06.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.893+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1000cc030 con 0x7fd1000cfa40 2023-11-24T06:29:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.893+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1000cc5f0 con 0x7fd1000cfa40 2023-11-24T06:29:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.894+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10008ba80 con 0x7fd1000cfa40 2023-11-24T06:29:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.895+0000 7fd0fcff9640 1 -- 172.21.15.64:0/1010390637 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0e80129d0 con 0x7fd1000cfa40 2023-11-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.896+0000 7fd0fcff9640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd0e006ae40 0x7fd0e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.896+0000 7fd0fcff9640 1 -- 172.21.15.64:0/1010390637 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0e8088ea0 con 0x7fd1000cfa40 2023-11-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.896+0000 7fd0ff7fe640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd0e006ae40 0x7fd0e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.897+0000 7fd0ff7fe640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd0e006ae40 0x7fd0e006d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0036000 tx=0x7fd0f003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:06.901+0000 7fd0fcff9640 1 -- 172.21.15.64:0/1010390637 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0e8054200 con 0x7fd1000cfa40 2023-11-24T06:29:07.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.095+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd100067450 con 0x7fd1000cfa40 2023-11-24T06:29:07.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.096+0000 7fd0fcff9640 1 -- 172.21.15.64:0/1010390637 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12833 (secure 0 0 0) 0x7fd0e8057820 con 0x7fd1000cfa40 2023-11-24T06:29:07.099 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:07.099 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:02.804558+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.098+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd0e006ae40 msgr2=0x7fd0e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd0e006ae40 0x7fd0e006d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0036000 tx=0x7fd0f003c040 comp rx=0 tx=0).stop 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1000cfa40 msgr2=0x7fd1000c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1000cfa40 0x7fd1000c8650 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800c450 tx=0x7fd0e800c910 comp rx=0 tx=0).stop 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 shutdown_connections 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd0e006ae40 0x7fd0e006d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd10015d3c0 0x7fd10006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1000cfa40 0x7fd1000c8650 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 --2- 172.21.15.64:0/1010390637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1000b4110 0x7fd1000cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 >> 172.21.15.64:0/1010390637 conn(0x7fd1000d3050 msgr2=0x7fd1000cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 shutdown_connections 2023-11-24T06:29:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.099+0000 7fd104d11640 1 -- 172.21.15.64:0/1010390637 wait complete. 2023-11-24T06:29:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:07.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:07.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.343+0000 7f4a1634e640 1 Processor -- start 2023-11-24T06:29:07.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.344+0000 7f4a1634e640 1 -- start start 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.345+0000 7f4a1634e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a1015d4b0 0x7f4a100d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.345+0000 7f4a1634e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.345+0000 7f4a1634e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a100d0320 con 0x7f4a100cddf0 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.345+0000 7f4a1634e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a100b2730 con 0x7f4a100cf9a0 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.345+0000 7f4a1634e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100d1ec0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a14b4b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a14b4b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56260/0 (socket says 172.21.15.64:56260) 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a14b4b640 1 -- 172.21.15.64:0/2161017249 learned_addr learned my addr 172.21.15.64:0/2161017249 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a1534c640 1 --2- 172.21.15.64:0/2161017249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a1015d4b0 0x7f4a100d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a14b4b640 1 -- 172.21.15.64:0/2161017249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a1015d4b0 msgr2=0x7f4a100d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:07.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a14b4b640 1 --2- 172.21.15.64:0/2161017249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a1015d4b0 0x7f4a100d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a14b4b640 1 -- 172.21.15.64:0/2161017249 >> v1:172.21.15.162:6789/0 conn(0x7f4a100cddf0 legacy=0x7f4a100ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:07.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.346+0000 7f4a14b4b640 1 -- 172.21.15.64:0/2161017249 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a100b29d0 con 0x7f4a100cf9a0 2023-11-24T06:29:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.347+0000 7f4a14b4b640 1 --2- 172.21.15.64:0/2161017249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100cf9a0 0x7f4a100cfde0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a000098a0 tx=0x7f4a0002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d42361056d07f1f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a07fff640 1 -- 172.21.15.64:0/2161017249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00031640 con 0x7f4a100cf9a0 2023-11-24T06:29:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a07fff640 1 -- 172.21.15.64:0/2161017249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000317a0 con 0x7f4a100cf9a0 2023-11-24T06:29:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a07fff640 1 -- 172.21.15.64:0/2161017249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00031b30 con 0x7f4a100cf9a0 2023-11-24T06:29:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a1634e640 1 -- 172.21.15.64:0/2161017249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100cf9a0 msgr2=0x7f4a100cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a1634e640 1 --2- 172.21.15.64:0/2161017249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100cf9a0 0x7f4a100cfde0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a000098a0 tx=0x7f4a0002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a1634e640 1 -- 172.21.15.64:0/2161017249 shutdown_connections 2023-11-24T06:29:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a1634e640 1 --2- 172.21.15.64:0/2161017249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a1634e640 1 --2- 172.21.15.64:0/2161017249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a1015d4b0 0x7f4a100d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.348+0000 7f4a1634e640 1 -- 172.21.15.64:0/2161017249 >> 172.21.15.64:0/2161017249 conn(0x7f4a100d3000 msgr2=0x7f4a100d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.349+0000 7f4a1634e640 1 -- 172.21.15.64:0/2161017249 shutdown_connections 2023-11-24T06:29:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.349+0000 7f4a1634e640 1 -- 172.21.15.64:0/2161017249 wait complete. 2023-11-24T06:29:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.351+0000 7f4a1634e640 1 Processor -- start 2023-11-24T06:29:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.351+0000 7f4a1634e640 1 -- start start 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.352+0000 7f4a1634e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a100cddf0 0x7f4a101f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.352+0000 7f4a1634e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a100cf9a0 0x7f4a100afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.352+0000 7f4a1634e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1015d4b0 0x7f4a100b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.352+0000 7f4a1634e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100b65f0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.352+0000 7f4a1634e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100b5cc0 con 0x7f4a100cf9a0 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.352+0000 7f4a1634e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100b5fa0 con 0x7f4a100cddf0 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1015d4b0 0x7f4a100b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1015d4b0 0x7f4a100b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56276/0 (socket says 172.21.15.64:56276) 2023-11-24T06:29:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 -- 172.21.15.64:0/404711990 learned_addr learned my addr 172.21.15.64:0/404711990 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a14b4b640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a100cf9a0 0x7f4a100afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 -- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a100cddf0 msgr2=0x7f4a101f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a1534c640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a100cddf0 0x7f4a101f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a100cddf0 0x7f4a101f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 -- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a100cf9a0 msgr2=0x7f4a100afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a100cf9a0 0x7f4a100afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 -- 172.21.15.64:0/404711990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000094d0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a14b4b640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a100cf9a0 0x7f4a100afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a1534c640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a100cddf0 0x7f4a101f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.353+0000 7f4a15b4d640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1015d4b0 0x7f4a100b02c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800c780 tx=0x7f4a0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.354+0000 7f4a05ffb640 1 -- 172.21.15.64:0/404711990 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08004b60 con 0x7f4a1015d4b0 2023-11-24T06:29:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.354+0000 7f4a05ffb640 1 -- 172.21.15.64:0/404711990 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0800ce90 con 0x7f4a1015d4b0 2023-11-24T06:29:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.354+0000 7f4a05ffb640 1 -- 172.21.15.64:0/404711990 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800a850 con 0x7f4a1015d4b0 2023-11-24T06:29:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.354+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1015da10 con 0x7f4a1015d4b0 2023-11-24T06:29:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.354+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a100b6a80 con 0x7f4a1015d4b0 2023-11-24T06:29:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.355+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a100b67a0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.356+0000 7f4a05ffb640 1 -- 172.21.15.64:0/404711990 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a0800a9b0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.357+0000 7f4a05ffb640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f49f406ae40 0x7f49f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.357+0000 7f4a05ffb640 1 -- 172.21.15.64:0/404711990 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4a080893f0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.357+0000 7f4a1534c640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f49f406ae40 0x7f49f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.358+0000 7f4a1534c640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f49f406ae40 0x7f49f406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006fd0 tx=0x7f4a0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.361+0000 7f4a05ffb640 1 -- 172.21.15.64:0/404711990 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a100b67a0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.545+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a100b67a0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.546+0000 7f4a05ffb640 1 -- 172.21.15.64:0/404711990 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a100b67a0 con 0x7f4a1015d4b0 2023-11-24T06:29:07.549 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:29:07.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.548+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f49f406ae40 msgr2=0x7f49f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.548+0000 7f4a1634e640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f49f406ae40 0x7f49f406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006fd0 tx=0x7f4a0c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1015d4b0 msgr2=0x7f4a100b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1015d4b0 0x7f4a100b02c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800c780 tx=0x7f4a0800cc40 comp rx=0 tx=0).stop 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 shutdown_connections 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f49f406ae40 0x7f49f406d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1015d4b0 0x7f4a100b02c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a100cf9a0 0x7f4a100afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 --2- 172.21.15.64:0/404711990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a100cddf0 0x7f4a101f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 >> 172.21.15.64:0/404711990 conn(0x7f4a100d3000 msgr2=0x7f4a100b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:07.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 shutdown_connections 2023-11-24T06:29:07.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.549+0000 7f4a1634e640 1 -- 172.21.15.64:0/404711990 wait complete. 2023-11-24T06:29:07.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:29:07.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.792+0000 7f6b21eff640 1 Processor -- start 2023-11-24T06:29:07.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.792+0000 7f6b21eff640 1 -- start start 2023-11-24T06:29:07.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b21eff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6b1c143c40 0x7f6b1c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b21eff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b1c0717d0 0x7f6b1c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b21eff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1c070e50 con 0x7f6b1c0cca40 2023-11-24T06:29:07.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b21eff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0723c0 con 0x7f6b1c0717d0 2023-11-24T06:29:07.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b21eff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0cce40 con 0x7f6b1c143c40 2023-11-24T06:29:07.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b1bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b1c0717d0 0x7f6b1c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b1bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b1c0717d0 0x7f6b1c0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56278/0 (socket says 172.21.15.64:56278) 2023-11-24T06:29:07.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b1bfff640 1 -- 172.21.15.64:0/2759561254 learned_addr learned my addr 172.21.15.64:0/2759561254 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:07.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.794+0000 7f6b20efd640 1 --2- 172.21.15.64:0/2759561254 >> v2:172.21.15.162:3300/0 conn(0x7f6b1c143c40 0x7f6b1c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.795+0000 7f6b1bfff640 1 -- 172.21.15.64:0/2759561254 >> v2:172.21.15.162:3300/0 conn(0x7f6b1c143c40 msgr2=0x7f6b1c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:07.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.795+0000 7f6b1bfff640 1 --2- 172.21.15.64:0/2759561254 >> v2:172.21.15.162:3300/0 conn(0x7f6b1c143c40 0x7f6b1c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.795+0000 7f6b1bfff640 1 -- 172.21.15.64:0/2759561254 >> v1:172.21.15.162:6789/0 conn(0x7f6b1c0cca40 legacy=0x7f6b1c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:07.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.795+0000 7f6b1bfff640 1 -- 172.21.15.64:0/2759561254 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c06e170 con 0x7f6b1c0717d0 2023-11-24T06:29:07.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.796+0000 7f6b1bfff640 1 --2- 172.21.15.64:0/2759561254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0cc500 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06e410 tx=0x7f6b0802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab17a14a8c67684 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.796+0000 7f6b1b7fe640 1 -- 172.21.15.64:0/2759561254 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080317e0 con 0x7f6b1c0717d0 2023-11-24T06:29:07.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.796+0000 7f6b1b7fe640 1 -- 172.21.15.64:0/2759561254 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08031940 con 0x7f6b1c0717d0 2023-11-24T06:29:07.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.796+0000 7f6b1b7fe640 1 -- 172.21.15.64:0/2759561254 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08031cd0 con 0x7f6b1c0717d0 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 -- 172.21.15.64:0/2759561254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 msgr2=0x7f6b1c0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 --2- 172.21.15.64:0/2759561254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0cc500 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06e410 tx=0x7f6b0802b880 comp rx=0 tx=0).stop 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 -- 172.21.15.64:0/2759561254 shutdown_connections 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 --2- 172.21.15.64:0/2759561254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0cc500 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 --2- 172.21.15.64:0/2759561254 >> v2:172.21.15.162:3300/0 conn(0x7f6b1c143c40 0x7f6b1c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 -- 172.21.15.64:0/2759561254 >> 172.21.15.64:0/2759561254 conn(0x7f6b1c06a800 msgr2=0x7f6b1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 -- 172.21.15.64:0/2759561254 shutdown_connections 2023-11-24T06:29:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.797+0000 7f6b21eff640 1 -- 172.21.15.64:0/2759561254 wait complete. 2023-11-24T06:29:07.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.799+0000 7f6b21eff640 1 Processor -- start 2023-11-24T06:29:07.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.799+0000 7f6b21eff640 1 -- start start 2023-11-24T06:29:07.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b21eff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b21eff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b1c0cca40 0x7f6b1c136b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b21eff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b1c143c40 0x7f6b1c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b21eff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b2f60 con 0x7f6b1c0717d0 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b21eff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b20d0 con 0x7f6b1c0cca40 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b21eff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b22e0 con 0x7f6b1c143c40 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b20efd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b20efd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56284/0 (socket says 172.21.15.64:56284) 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.800+0000 7f6b20efd640 1 -- 172.21.15.64:0/239487579 learned_addr learned my addr 172.21.15.64:0/239487579 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b1bfff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b1c0cca40 0x7f6b1c136b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b216fe640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b1c143c40 0x7f6b1c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b20efd640 1 -- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b1c143c40 msgr2=0x7f6b1c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b20efd640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b1c143c40 0x7f6b1c0b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b20efd640 1 -- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b1c0cca40 msgr2=0x7f6b1c136b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b20efd640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b1c0cca40 0x7f6b1c136b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b20efd640 1 -- 172.21.15.64:0/239487579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080094d0 con 0x7f6b1c0717d0 2023-11-24T06:29:07.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b1bfff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b1c0cca40 0x7f6b1c136b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:07.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b216fe640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b1c143c40 0x7f6b1c0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:07.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b20efd640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0b3380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000d5f0 tx=0x7f6b1000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b197fa640 1 -- 172.21.15.64:0/239487579 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100125f0 con 0x7f6b1c0717d0 2023-11-24T06:29:07.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b197fa640 1 -- 172.21.15.64:0/239487579 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b10012750 con 0x7f6b1c0717d0 2023-11-24T06:29:07.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b197fa640 1 -- 172.21.15.64:0/239487579 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10012b00 con 0x7f6b1c0717d0 2023-11-24T06:29:07.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c0b25c0 con 0x7f6b1c0717d0 2023-11-24T06:29:07.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.801+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c0b2b80 con 0x7f6b1c0717d0 2023-11-24T06:29:07.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.803+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c0b31d0 con 0x7f6b1c0717d0 2023-11-24T06:29:07.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.804+0000 7f6b197fa640 1 -- 172.21.15.64:0/239487579 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b10012c60 con 0x7f6b1c0717d0 2023-11-24T06:29:07.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.804+0000 7f6b197fa640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b0006ae40 0x7f6b0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.804+0000 7f6b197fa640 1 -- 172.21.15.64:0/239487579 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6b10089830 con 0x7f6b1c0717d0 2023-11-24T06:29:07.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.805+0000 7f6b1bfff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b0006ae40 0x7f6b0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.805+0000 7f6b1bfff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b0006ae40 0x7f6b0006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b08036000 tx=0x7f6b0803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:07.809+0000 7f6b197fa640 1 -- 172.21.15.64:0/239487579 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b10054b90 con 0x7f6b1c0717d0 2023-11-24T06:29:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.007+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6b1c067450 con 0x7f6b1c0717d0 2023-11-24T06:29:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.008+0000 7f6b197fa640 1 -- 172.21.15.64:0/239487579 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6b100581b0 con 0x7f6b1c0717d0 2023-11-24T06:29:08.011 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:08.011 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T06:29:08.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:08.012 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T06:29:08.012 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T06:29:08.012 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:08.012 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b0006ae40 msgr2=0x7f6b0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b0006ae40 0x7f6b0006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b08036000 tx=0x7f6b0803d040 comp rx=0 tx=0).stop 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 msgr2=0x7f6b1c0b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0b3380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000d5f0 tx=0x7f6b1000dab0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 shutdown_connections 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6b0006ae40 0x7f6b0006d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b1c143c40 0x7f6b1c0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b1c0cca40 0x7f6b1c136b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 --2- 172.21.15.64:0/239487579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c0717d0 0x7f6b1c0b3380 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 >> 172.21.15.64:0/239487579 conn(0x7f6b1c06a800 msgr2=0x7f6b1c13f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 shutdown_connections 2023-11-24T06:29:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.011+0000 7f6b21eff640 1 -- 172.21.15.64:0/239487579 wait complete. 2023-11-24T06:29:08.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:29:08.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:29:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.261+0000 7fd059fdb640 1 Processor -- start 2023-11-24T06:29:08.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.262+0000 7fd059fdb640 1 -- start start 2023-11-24T06:29:08.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.263+0000 7fd059fdb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd0540d0100 0x7fd0540b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.263+0000 7fd059fdb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0540d0f40 0x7fd0540d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.263+0000 7fd059fdb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0540d0a10 con 0x7fd054160920 2023-11-24T06:29:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.263+0000 7fd059fdb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540cf080 con 0x7fd0540d0100 2023-11-24T06:29:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.263+0000 7fd059fdb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540c6be0 con 0x7fd0540d0f40 2023-11-24T06:29:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd053fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd0540d0100 0x7fd0540b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd053fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd0540d0100 0x7fd0540b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56294/0 (socket says 172.21.15.64:56294) 2023-11-24T06:29:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd053fff640 1 -- 172.21.15.64:0/1873834089 learned_addr learned my addr 172.21.15.64:0/1873834089 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd0597da640 1 --2- 172.21.15.64:0/1873834089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0540d0f40 0x7fd0540d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd053fff640 1 -- 172.21.15.64:0/1873834089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0540d0f40 msgr2=0x7fd0540d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd053fff640 1 --2- 172.21.15.64:0/1873834089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0540d0f40 0x7fd0540d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd053fff640 1 -- 172.21.15.64:0/1873834089 >> v1:172.21.15.162:6789/0 conn(0x7fd054160920 legacy=0x7fd0540cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.264+0000 7fd053fff640 1 -- 172.21.15.64:0/1873834089 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540ce7e0 con 0x7fd0540d0100 2023-11-24T06:29:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.266+0000 7fd053fff640 1 --2- 172.21.15.64:0/1873834089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd0540b3f30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd048009870 tx=0x7fd04802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6903156eeeddb30c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.266+0000 7fd0537fe640 1 -- 172.21.15.64:0/1873834089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048031640 con 0x7fd0540d0100 2023-11-24T06:29:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.266+0000 7fd0537fe640 1 -- 172.21.15.64:0/1873834089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0480317a0 con 0x7fd0540d0100 2023-11-24T06:29:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.266+0000 7fd0537fe640 1 -- 172.21.15.64:0/1873834089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048031b30 con 0x7fd0540d0100 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 -- 172.21.15.64:0/1873834089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 msgr2=0x7fd0540b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 --2- 172.21.15.64:0/1873834089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd0540b3f30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd048009870 tx=0x7fd04802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 -- 172.21.15.64:0/1873834089 shutdown_connections 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 --2- 172.21.15.64:0/1873834089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0540d0f40 0x7fd0540d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 --2- 172.21.15.64:0/1873834089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd0540b3f30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 -- 172.21.15.64:0/1873834089 >> 172.21.15.64:0/1873834089 conn(0x7fd0540d3090 msgr2=0x7fd0540d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 -- 172.21.15.64:0/1873834089 shutdown_connections 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.267+0000 7fd059fdb640 1 -- 172.21.15.64:0/1873834089 wait complete. 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.269+0000 7fd059fdb640 1 Processor -- start 2023-11-24T06:29:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.270+0000 7fd059fdb640 1 -- start start 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd059fdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd054151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd059fdb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0540d0f40 0x7fd0540ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd059fdb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd054160920 0x7fd0540b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd059fdb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540caa20 con 0x7fd0540d0100 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd059fdb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540cad00 con 0x7fd0540d0f40 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd059fdb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540cafe0 con 0x7fd054160920 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd058fd9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd054151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd058fd9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd054151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56298/0 (socket says 172.21.15.64:56298) 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.271+0000 7fd058fd9640 1 -- 172.21.15.64:0/102372534 learned_addr learned my addr 172.21.15.64:0/102372534 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd0597da640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd054160920 0x7fd0540b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd053fff640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0540d0f40 0x7fd0540ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd058fd9640 1 -- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd054160920 msgr2=0x7fd0540b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd058fd9640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd054160920 0x7fd0540b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd058fd9640 1 -- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0540d0f40 msgr2=0x7fd0540ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd058fd9640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0540d0f40 0x7fd0540ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd058fd9640 1 -- 172.21.15.64:0/102372534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480094d0 con 0x7fd0540d0100 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd053fff640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0540d0f40 0x7fd0540ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd058fd9640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd054151580 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd04400b3e0 tx=0x7fd04400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.272+0000 7fd0517fa640 1 -- 172.21.15.64:0/102372534 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044011830 con 0x7fd0540d0100 2023-11-24T06:29:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.273+0000 7fd0517fa640 1 -- 172.21.15.64:0/102372534 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0440119b0 con 0x7fd0540d0100 2023-11-24T06:29:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.273+0000 7fd0517fa640 1 -- 172.21.15.64:0/102372534 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044011d60 con 0x7fd0540d0100 2023-11-24T06:29:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.273+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0540cb2c0 con 0x7fd0540d0100 2023-11-24T06:29:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.273+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0540cbac0 con 0x7fd0540d0100 2023-11-24T06:29:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.274+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd054067450 con 0x7fd0540d0100 2023-11-24T06:29:08.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.275+0000 7fd0517fa640 1 -- 172.21.15.64:0/102372534 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd04401a400 con 0x7fd0540d0100 2023-11-24T06:29:08.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.276+0000 7fd0517fa640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd04006ae40 0x7fd04006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.276+0000 7fd0517fa640 1 -- 172.21.15.64:0/102372534 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd044089390 con 0x7fd0540d0100 2023-11-24T06:29:08.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.276+0000 7fd053fff640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd04006ae40 0x7fd04006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.277+0000 7fd053fff640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd04006ae40 0x7fd04006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd048036000 tx=0x7fd04803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:08.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.282+0000 7fd0517fa640 1 -- 172.21.15.64:0/102372534 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd044054820 con 0x7fd0540d0100 2023-11-24T06:29:08.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:08.485+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd05408c7e0 con 0x7fd0540d0100 2023-11-24T06:29:09.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.227+0000 7fd0517fa640 1 -- 172.21.15.64:0/102372534 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd0440150b0 con 0x7fd0540d0100 2023-11-24T06:29:09.230 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T06:29:09.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd04006ae40 msgr2=0x7fd04006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd04006ae40 0x7fd04006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd048036000 tx=0x7fd04803c040 comp rx=0 tx=0).stop 2023-11-24T06:29:09.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 msgr2=0x7fd054151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd054151580 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd04400b3e0 tx=0x7fd04400b8a0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 shutdown_connections 2023-11-24T06:29:09.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd04006ae40 0x7fd04006d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd054160920 0x7fd0540b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0540d0f40 0x7fd0540ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 --2- 172.21.15.64:0/102372534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0540d0100 0x7fd054151580 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 >> 172.21.15.64:0/102372534 conn(0x7fd0540d3090 msgr2=0x7fd05406c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:09.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.231+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 shutdown_connections 2023-11-24T06:29:09.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.232+0000 7fd059fdb640 1 -- 172.21.15.64:0/102372534 wait complete. 2023-11-24T06:29:09.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:29:09.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.484+0000 7f6a2b59e640 1 Processor -- start 2023-11-24T06:29:09.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.484+0000 7f6a2b59e640 1 -- start start 2023-11-24T06:29:09.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.486+0000 7f6a2b59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6a2c143c40 0x7f6a2c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.486+0000 7f6a2b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c0ccc10 0x7f6a2c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.486+0000 7f6a2b59e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2c070e50 con 0x7f6a2c0717d0 2023-11-24T06:29:09.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.486+0000 7f6a2b59e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2c0723c0 con 0x7f6a2c143c40 2023-11-24T06:29:09.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.486+0000 7f6a2b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c144020 con 0x7f6a2c0ccc10 2023-11-24T06:29:09.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.487+0000 7f6a2a59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6a2c143c40 0x7f6a2c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.487+0000 7f6a23fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6a2c0717d0 0x7f6a2c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:59746/0 (socket says 172.21.15.64:59746) 2023-11-24T06:29:09.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.487+0000 7f6a23fff640 1 -- 172.21.15.64:0/3247681833 learned_addr learned my addr 172.21.15.64:0/3247681833 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:09.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.487+0000 7f6a2ad9d640 1 --2- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c0ccc10 0x7f6a2c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.487+0000 7f6a2a59c640 1 -- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c0ccc10 msgr2=0x7f6a2c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:09.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.487+0000 7f6a2a59c640 1 --2- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c0ccc10 0x7f6a2c06d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.488+0000 7f6a2a59c640 1 -- 172.21.15.64:0/3247681833 >> v1:172.21.15.162:6789/0 conn(0x7f6a2c0717d0 legacy=0x7f6a2c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.488+0000 7f6a2a59c640 1 -- 172.21.15.64:0/3247681833 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c071e00 con 0x7f6a2c143c40 2023-11-24T06:29:09.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.489+0000 7f6a2a59c640 1 --2- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c143c40 0x7f6a2c071290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097a0 tx=0x7f6a1c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d675e94ed47c4a4f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:09.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a29d9b640 1 -- 172.21.15.64:0/3247681833 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c031780 con 0x7f6a2c143c40 2023-11-24T06:29:09.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a29d9b640 1 -- 172.21.15.64:0/3247681833 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0318e0 con 0x7f6a2c143c40 2023-11-24T06:29:09.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a29d9b640 1 -- 172.21.15.64:0/3247681833 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c031c70 con 0x7f6a2c143c40 2023-11-24T06:29:09.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 -- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c143c40 msgr2=0x7f6a2c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c143c40 0x7f6a2c071290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097a0 tx=0x7f6a1c02b820 comp rx=0 tx=0).stop 2023-11-24T06:29:09.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 -- 172.21.15.64:0/3247681833 shutdown_connections 2023-11-24T06:29:09.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c0ccc10 0x7f6a2c06d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/3247681833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c143c40 0x7f6a2c071290 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 -- 172.21.15.64:0/3247681833 >> 172.21.15.64:0/3247681833 conn(0x7f6a2c06a800 msgr2=0x7f6a2c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:09.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 -- 172.21.15.64:0/3247681833 shutdown_connections 2023-11-24T06:29:09.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.490+0000 7f6a2b59e640 1 -- 172.21.15.64:0/3247681833 wait complete. 2023-11-24T06:29:09.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.492+0000 7f6a2b59e640 1 Processor -- start 2023-11-24T06:29:09.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.492+0000 7f6a2b59e640 1 -- start start 2023-11-24T06:29:09.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.493+0000 7f6a2b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a2c0717d0 0x7f6a2c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.493+0000 7f6a2b59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c0ccc10 0x7f6a2c0c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.493+0000 7f6a2b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c143c40 0x7f6a2c12fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.493+0000 7f6a2b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c0c6c60 con 0x7f6a2c0717d0 2023-11-24T06:29:09.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.493+0000 7f6a2b59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c0c6f40 con 0x7f6a2c0ccc10 2023-11-24T06:29:09.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.493+0000 7f6a2b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c0c7220 con 0x7f6a2c143c40 2023-11-24T06:29:09.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a2c0717d0 0x7f6a2c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a2c0717d0 0x7f6a2c15a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56314/0 (socket says 172.21.15.64:56314) 2023-11-24T06:29:09.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 -- 172.21.15.64:0/2646658900 learned_addr learned my addr 172.21.15.64:0/2646658900 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:09.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a23fff640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c0ccc10 0x7f6a2c0c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2ad9d640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c143c40 0x7f6a2c12fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 -- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c143c40 msgr2=0x7f6a2c12fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c143c40 0x7f6a2c12fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 -- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c0ccc10 msgr2=0x7f6a2c0c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c0ccc10 0x7f6a2c0c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.494+0000 7f6a2a59c640 1 -- 172.21.15.64:0/2646658900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c009400 con 0x7f6a2c0717d0 2023-11-24T06:29:09.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.495+0000 7f6a2a59c640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a2c0717d0 0x7f6a2c15a4c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c034040 tx=0x7f6a1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:09.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.495+0000 7f6a22ffd640 1 -- 172.21.15.64:0/2646658900 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c032820 con 0x7f6a2c0717d0 2023-11-24T06:29:09.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.495+0000 7f6a22ffd640 1 -- 172.21.15.64:0/2646658900 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c032980 con 0x7f6a2c0717d0 2023-11-24T06:29:09.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.495+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2c0c7500 con 0x7f6a2c0717d0 2023-11-24T06:29:09.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.495+0000 7f6a22ffd640 1 -- 172.21.15.64:0/2646658900 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c032d30 con 0x7f6a2c0717d0 2023-11-24T06:29:09.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.495+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2c15aa00 con 0x7f6a2c0717d0 2023-11-24T06:29:09.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.496+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2c0c7a20 con 0x7f6a2c0717d0 2023-11-24T06:29:09.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.497+0000 7f6a22ffd640 1 -- 172.21.15.64:0/2646658900 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a1c04e440 con 0x7f6a2c0717d0 2023-11-24T06:29:09.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.498+0000 7f6a22ffd640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a0c06ae40 0x7f6a0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.498+0000 7f6a22ffd640 1 -- 172.21.15.64:0/2646658900 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6a1c0ab4f0 con 0x7f6a2c0717d0 2023-11-24T06:29:09.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.499+0000 7f6a23fff640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a0c06ae40 0x7f6a0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.499+0000 7f6a23fff640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a0c06ae40 0x7f6a0c06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0cb5d0 tx=0x7f6a14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:09.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.504+0000 7f6a22ffd640 1 -- 172.21.15.64:0/2646658900 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a1c076950 con 0x7f6a2c0717d0 2023-11-24T06:29:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:09.699+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6a2c067450 con 0x7f6a2c0717d0 2023-11-24T06:29:10.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.245+0000 7f6a22ffd640 1 -- 172.21.15.64:0/2646658900 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6a1c079f70 con 0x7f6a2c0717d0 2023-11-24T06:29:10.248 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T06:29:10.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a0c06ae40 msgr2=0x7f6a0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a0c06ae40 0x7f6a0c06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0cb5d0 tx=0x7f6a14009040 comp rx=0 tx=0).stop 2023-11-24T06:29:10.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a2c0717d0 msgr2=0x7f6a2c15a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a2c0717d0 0x7f6a2c15a4c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c034040 tx=0x7f6a1c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 shutdown_connections 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6a0c06ae40 0x7f6a0c06d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a2c143c40 0x7f6a2c12fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.249+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a2c0ccc10 0x7f6a2c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.250+0000 7f6a2b59e640 1 --2- 172.21.15.64:0/2646658900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a2c0717d0 0x7f6a2c15a4c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.250+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 >> 172.21.15.64:0/2646658900 conn(0x7f6a2c06a800 msgr2=0x7f6a2c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.250+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 shutdown_connections 2023-11-24T06:29:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.250+0000 7f6a2b59e640 1 -- 172.21.15.64:0/2646658900 wait complete. 2023-11-24T06:29:10.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:29:10.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.478+0000 7f29c4a45640 1 Processor -- start 2023-11-24T06:29:10.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.478+0000 7f29c4a45640 1 -- start start 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29c4a45640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29c013cf10 0x7f29c013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29c4a45640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29c013fae0 0x7f29c00cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29c4a45640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c00ce440 con 0x7f29c00cdc20 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29c4a45640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c013ff90 con 0x7f29c013cf10 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29c4a45640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00ce720 con 0x7f29c013fae0 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29bed9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29c013cf10 0x7f29c013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29bed9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29c013cf10 0x7f29c013f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56326/0 (socket says 172.21.15.64:56326) 2023-11-24T06:29:10.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.480+0000 7f29bed9d640 1 -- 172.21.15.64:0/1862259221 learned_addr learned my addr 172.21.15.64:0/1862259221 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:10.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.481+0000 7f29b7fff640 1 --2- 172.21.15.64:0/1862259221 >> v2:172.21.15.162:3300/0 conn(0x7f29c013fae0 0x7f29c00cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.481+0000 7f29bed9d640 1 -- 172.21.15.64:0/1862259221 >> v2:172.21.15.162:3300/0 conn(0x7f29c013fae0 msgr2=0x7f29c00cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:10.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.481+0000 7f29bed9d640 1 --2- 172.21.15.64:0/1862259221 >> v2:172.21.15.162:3300/0 conn(0x7f29c013fae0 0x7f29c00cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.481+0000 7f29bed9d640 1 -- 172.21.15.64:0/1862259221 >> v1:172.21.15.64:6789/0 conn(0x7f29c00cdc20 legacy=0x7f29c015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.481+0000 7f29bed9d640 1 -- 172.21.15.64:0/1862259221 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00cea00 con 0x7f29c013cf10 2023-11-24T06:29:10.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.482+0000 7f29bed9d640 1 --2- 172.21.15.64:0/1862259221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c013cf10 0x7f29c013f5a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f29c006ffb0 tx=0x7f29b002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebd25f4ed1a1b79 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.482+0000 7f29be59c640 1 -- 172.21.15.64:0/1862259221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0030860 con 0x7f29c013cf10 2023-11-24T06:29:10.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.482+0000 7f29be59c640 1 -- 172.21.15.64:0/1862259221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00309c0 con 0x7f29c013cf10 2023-11-24T06:29:10.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.482+0000 7f29be59c640 1 -- 172.21.15.64:0/1862259221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0030d50 con 0x7f29c013cf10 2023-11-24T06:29:10.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 -- 172.21.15.64:0/1862259221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c013cf10 msgr2=0x7f29c013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 --2- 172.21.15.64:0/1862259221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c013cf10 0x7f29c013f5a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f29c006ffb0 tx=0x7f29b002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 -- 172.21.15.64:0/1862259221 shutdown_connections 2023-11-24T06:29:10.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 --2- 172.21.15.64:0/1862259221 >> v2:172.21.15.162:3300/0 conn(0x7f29c013fae0 0x7f29c00cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 --2- 172.21.15.64:0/1862259221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c013cf10 0x7f29c013f5a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 -- 172.21.15.64:0/1862259221 >> 172.21.15.64:0/1862259221 conn(0x7f29c006a7d0 msgr2=0x7f29c006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:10.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 -- 172.21.15.64:0/1862259221 shutdown_connections 2023-11-24T06:29:10.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.483+0000 7f29c4a45640 1 -- 172.21.15.64:0/1862259221 wait complete. 2023-11-24T06:29:10.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.485+0000 7f29c4a45640 1 Processor -- start 2023-11-24T06:29:10.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.485+0000 7f29c4a45640 1 -- start start 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29c4a45640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c00cdc20 0x7f29c0138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29c4a45640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c013fae0 0x7f29c00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29c4a45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c00ace10 0x7f29c0139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29c4a45640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0138cc0 con 0x7f29c00ace10 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29c4a45640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00adfa0 con 0x7f29c013fae0 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29c4a45640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00ad390 con 0x7f29c00cdc20 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29bf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c00ace10 0x7f29c0139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29bf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c00ace10 0x7f29c0139c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56330/0 (socket says 172.21.15.64:56330) 2023-11-24T06:29:10.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.486+0000 7f29bf59e640 1 -- 172.21.15.64:0/345196531 learned_addr learned my addr 172.21.15.64:0/345196531 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29b7fff640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c013fae0 0x7f29c00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bed9d640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c00cdc20 0x7f29c0138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bf59e640 1 -- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c00cdc20 msgr2=0x7f29c0138780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bf59e640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c00cdc20 0x7f29c0138780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bf59e640 1 -- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c013fae0 msgr2=0x7f29c00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bf59e640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c013fae0 0x7f29c00ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bf59e640 1 -- 172.21.15.64:0/345196531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00094d0 con 0x7f29c00ace10 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29b7fff640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c013fae0 0x7f29c00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bed9d640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c00cdc20 0x7f29c0138780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:10.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.487+0000 7f29bf59e640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c00ace10 0x7f29c0139c70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c7d0 tx=0x7f29b800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.488+0000 7f29b77fe640 1 -- 172.21.15.64:0/345196531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8003b70 con 0x7f29c00ace10 2023-11-24T06:29:10.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.488+0000 7f29b77fe640 1 -- 172.21.15.64:0/345196531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b8003cf0 con 0x7f29c00ace10 2023-11-24T06:29:10.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.488+0000 7f29b77fe640 1 -- 172.21.15.64:0/345196531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800aa90 con 0x7f29c00ace10 2023-11-24T06:29:10.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.488+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c00ad670 con 0x7f29c00ace10 2023-11-24T06:29:10.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.488+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00adc30 con 0x7f29c00ace10 2023-11-24T06:29:10.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.489+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c00b19d0 con 0x7f29c00ace10 2023-11-24T06:29:10.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.490+0000 7f29b77fe640 1 -- 172.21.15.64:0/345196531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29b800abf0 con 0x7f29c00ace10 2023-11-24T06:29:10.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.491+0000 7f29b77fe640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f29a006ae70 0x7f29a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.491+0000 7f29b77fe640 1 -- 172.21.15.64:0/345196531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f29b8089d90 con 0x7f29c00ace10 2023-11-24T06:29:10.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.491+0000 7f29bed9d640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f29a006ae70 0x7f29a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.492+0000 7f29bed9d640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f29a006ae70 0x7f29a006d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29b0036000 tx=0x7f29b003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.495+0000 7f29b77fe640 1 -- 172.21.15.64:0/345196531 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29b8055010 con 0x7f29c00ace10 2023-11-24T06:29:10.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:10.722+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f29c0067450 con 0x7f29c00ace10 2023-11-24T06:29:11.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.270+0000 7f29b77fe640 1 -- 172.21.15.64:0/345196531 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f29b8003e60 con 0x7f29c00ace10 2023-11-24T06:29:11.279 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T06:29:11.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f29a006ae70 msgr2=0x7f29a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f29a006ae70 0x7f29a006d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29b0036000 tx=0x7f29b003b040 comp rx=0 tx=0).stop 2023-11-24T06:29:11.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c00ace10 msgr2=0x7f29c0139c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c00ace10 0x7f29c0139c70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c7d0 tx=0x7f29b800cc90 comp rx=0 tx=0).stop 2023-11-24T06:29:11.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 shutdown_connections 2023-11-24T06:29:11.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f29a006ae70 0x7f29a006d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c00ace10 0x7f29c0139c70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c013fae0 0x7f29c00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 --2- 172.21.15.64:0/345196531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c00cdc20 0x7f29c0138780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.280+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 >> 172.21.15.64:0/345196531 conn(0x7f29c006a7d0 msgr2=0x7f29c00afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.282+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 shutdown_connections 2023-11-24T06:29:11.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.282+0000 7f29c4a45640 1 -- 172.21.15.64:0/345196531 wait complete. 2023-11-24T06:29:11.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:11.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.505+0000 7f7cce07c640 1 Processor -- start 2023-11-24T06:29:11.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.505+0000 7f7cce07c640 1 -- start start 2023-11-24T06:29:11.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.507+0000 7f7cce07c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7cc813e360 0x7f7cc80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.507+0000 7f7cce07c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.507+0000 7f7cce07c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc815c5f0 con 0x7f7cc813df80 2023-11-24T06:29:11.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.507+0000 7f7cce07c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc806c470 con 0x7f7cc813e360 2023-11-24T06:29:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.507+0000 7f7cce07c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc806c750 con 0x7f7cc815f9c0 2023-11-24T06:29:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.507+0000 7f7cc7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.507+0000 7f7cc6ffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7cc813e360 0x7f7cc80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.508+0000 7f7cc7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:55352/0 (socket says 172.21.15.64:55352) 2023-11-24T06:29:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.508+0000 7f7cc7fff640 1 -- 172.21.15.64:0/1915441917 learned_addr learned my addr 172.21.15.64:0/1915441917 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.508+0000 7f7cc6ffd640 1 -- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 msgr2=0x7f7cc80cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.508+0000 7f7cc6ffd640 1 --2- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.508+0000 7f7cc6ffd640 1 -- 172.21.15.64:0/1915441917 >> v1:172.21.15.162:6789/0 conn(0x7f7cc813df80 legacy=0x7f7cc815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.508+0000 7f7cc6ffd640 1 -- 172.21.15.64:0/1915441917 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc806ca30 con 0x7f7cc813e360 2023-11-24T06:29:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.508+0000 7f7cc7fff640 1 --2- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:11.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.510+0000 7f7cc6ffd640 1 --2- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 0x7f7cc80cf290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00097a0 tx=0x7f7cb002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=532b0efe131fab54 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.510+0000 7f7cc67fc640 1 -- 172.21.15.64:0/1915441917 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb002d040 con 0x7f7cc813e360 2023-11-24T06:29:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.510+0000 7f7cc67fc640 1 -- 172.21.15.64:0/1915441917 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb0031810 con 0x7f7cc813e360 2023-11-24T06:29:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cc67fc640 1 -- 172.21.15.64:0/1915441917 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0031ba0 con 0x7f7cc813e360 2023-11-24T06:29:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 -- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 msgr2=0x7f7cc80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 --2- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 0x7f7cc80cf290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00097a0 tx=0x7f7cb002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 -- 172.21.15.64:0/1915441917 shutdown_connections 2023-11-24T06:29:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 --2- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 --2- 172.21.15.64:0/1915441917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 0x7f7cc80cf290 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 -- 172.21.15.64:0/1915441917 >> 172.21.15.64:0/1915441917 conn(0x7f7cc806a800 msgr2=0x7f7cc806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 -- 172.21.15.64:0/1915441917 shutdown_connections 2023-11-24T06:29:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.511+0000 7f7cce07c640 1 -- 172.21.15.64:0/1915441917 wait complete. 2023-11-24T06:29:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.513+0000 7f7cce07c640 1 Processor -- start 2023-11-24T06:29:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.513+0000 7f7cce07c640 1 -- start start 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cce07c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc813df80 0x7f7cc8133620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cce07c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 0x7f7cc814abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cce07c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cce07c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc814f850 con 0x7f7cc813df80 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cce07c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc8133b60 con 0x7f7cc813e360 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cce07c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc814b5f0 con 0x7f7cc815f9c0 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cc77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc813df80 0x7f7cc8133620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cc77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc813df80 0x7f7cc8133620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56344/0 (socket says 172.21.15.64:56344) 2023-11-24T06:29:11.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.514+0000 7f7cc77fe640 1 -- 172.21.15.64:0/2854964513 learned_addr learned my addr 172.21.15.64:0/2854964513 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc6ffd640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 0x7f7cc814abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc7fff640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc77fe640 1 -- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 msgr2=0x7f7cc80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc77fe640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc77fe640 1 -- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 msgr2=0x7f7cc814abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc77fe640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 0x7f7cc814abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc77fe640 1 -- 172.21.15.64:0/2854964513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00094d0 con 0x7f7cc813df80 2023-11-24T06:29:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.515+0000 7f7cc77fe640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc813df80 0x7f7cc8133620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c6c0 tx=0x7f7cb800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.516+0000 7f7cbffff640 1 -- 172.21.15.64:0/2854964513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8003b90 con 0x7f7cc813df80 2023-11-24T06:29:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.516+0000 7f7cbffff640 1 -- 172.21.15.64:0/2854964513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb8003cf0 con 0x7f7cc813df80 2023-11-24T06:29:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.516+0000 7f7cbffff640 1 -- 172.21.15.64:0/2854964513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8005310 con 0x7f7cc813df80 2023-11-24T06:29:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.516+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb0009400 con 0x7f7cc813df80 2023-11-24T06:29:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.516+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc814bbb0 con 0x7f7cc813df80 2023-11-24T06:29:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.517+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc80675d0 con 0x7f7cc813df80 2023-11-24T06:29:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.518+0000 7f7cbffff640 1 -- 172.21.15.64:0/2854964513 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7cb8005470 con 0x7f7cc813df80 2023-11-24T06:29:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.519+0000 7f7cbffff640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7ca806ae40 0x7f7ca806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.519+0000 7f7cbffff640 1 -- 172.21.15.64:0/2854964513 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cb8089470 con 0x7f7cc813df80 2023-11-24T06:29:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.519+0000 7f7cc6ffd640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7ca806ae40 0x7f7ca806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.520+0000 7f7cc6ffd640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7ca806ae40 0x7f7ca806d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0036000 tx=0x7f7cb003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.523+0000 7f7cbffff640 1 -- 172.21.15.64:0/2854964513 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7cb8054480 con 0x7f7cc813df80 2023-11-24T06:29:11.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.715+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cc806df00 con 0x7f7cc813df80 2023-11-24T06:29:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.716+0000 7f7cbffff640 1 -- 172.21.15.64:0/2854964513 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16558 (secure 0 0 0) 0x7f7cb8003e60 con 0x7f7cc813df80 2023-11-24T06:29:11.720 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:11.720 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:10.723839+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7ca806ae40 msgr2=0x7f7ca806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7ca806ae40 0x7f7ca806d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0036000 tx=0x7f7cb003c040 comp rx=0 tx=0).stop 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc813df80 msgr2=0x7f7cc8133620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc813df80 0x7f7cc8133620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c6c0 tx=0x7f7cb800cb80 comp rx=0 tx=0).stop 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 shutdown_connections 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7ca806ae40 0x7f7ca806d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7cc815f9c0 0x7f7cc80b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc813e360 0x7f7cc814abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 --2- 172.21.15.64:0/2854964513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc813df80 0x7f7cc8133620 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 >> 172.21.15.64:0/2854964513 conn(0x7f7cc806a800 msgr2=0x7f7cc815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 shutdown_connections 2023-11-24T06:29:11.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.719+0000 7f7cce07c640 1 -- 172.21.15.64:0/2854964513 wait complete. 2023-11-24T06:29:11.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.953+0000 7fce926ca640 1 Processor -- start 2023-11-24T06:29:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.953+0000 7fce926ca640 1 -- start start 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce926ca640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce8c15afb0 0x7fce8c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce926ca640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce8c13fae0 0x7fce8c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce926ca640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c13ff30 con 0x7fce8c13cfa0 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce926ca640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c0cdc40 con 0x7fce8c13fae0 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce926ca640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c0cdec0 con 0x7fce8c15afb0 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce916c8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fce8c13cfa0 0x7fce8c0cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33084/0 (socket says 172.21.15.64:33084) 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce916c8640 1 -- 172.21.15.64:0/3349906639 learned_addr learned my addr 172.21.15.64:0/3349906639 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.955+0000 7fce91ec9640 1 --2- 172.21.15.64:0/3349906639 >> v2:172.21.15.64:3300/0 conn(0x7fce8c13fae0 0x7fce8c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.956+0000 7fce7ffff640 1 -- 172.21.15.64:0/3349906639 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524299602 0 0) 0x7fce88001110 con 0x7fce8c13cfa0 2023-11-24T06:29:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.956+0000 7fce7ffff640 1 -- 172.21.15.64:0/3349906639 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce78003520 con 0x7fce8c13cfa0 2023-11-24T06:29:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.956+0000 7fce91ec9640 1 -- 172.21.15.64:0/3349906639 >> v2:172.21.15.162:3300/0 conn(0x7fce8c15afb0 msgr2=0x7fce8c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.956+0000 7fce91ec9640 1 --2- 172.21.15.64:0/3349906639 >> v2:172.21.15.162:3300/0 conn(0x7fce8c15afb0 0x7fce8c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.956+0000 7fce91ec9640 1 -- 172.21.15.64:0/3349906639 >> v1:172.21.15.64:6789/0 conn(0x7fce8c13cfa0 legacy=0x7fce8c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.956+0000 7fce91ec9640 1 -- 172.21.15.64:0/3349906639 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0ce1a0 con 0x7fce8c13fae0 2023-11-24T06:29:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.957+0000 7fce91ec9640 1 --2- 172.21.15.64:0/3349906639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13fae0 0x7fce8c0b2040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0b3fd0 tx=0x7fce8402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0731a5ec7c7b32e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.957+0000 7fce7ffff640 1 -- 172.21.15.64:0/3349906639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8402ee00 con 0x7fce8c13fae0 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.957+0000 7fce7ffff640 1 -- 172.21.15.64:0/3349906639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce84036440 con 0x7fce8c13fae0 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.957+0000 7fce7ffff640 1 -- 172.21.15.64:0/3349906639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce840367d0 con 0x7fce8c13fae0 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 -- 172.21.15.64:0/3349906639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13fae0 msgr2=0x7fce8c0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 --2- 172.21.15.64:0/3349906639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13fae0 0x7fce8c0b2040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0b3fd0 tx=0x7fce8402e050 comp rx=0 tx=0).stop 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 -- 172.21.15.64:0/3349906639 shutdown_connections 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 --2- 172.21.15.64:0/3349906639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13fae0 0x7fce8c0b2040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 --2- 172.21.15.64:0/3349906639 >> v2:172.21.15.162:3300/0 conn(0x7fce8c15afb0 0x7fce8c13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 -- 172.21.15.64:0/3349906639 >> 172.21.15.64:0/3349906639 conn(0x7fce8c06a800 msgr2=0x7fce8c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 -- 172.21.15.64:0/3349906639 shutdown_connections 2023-11-24T06:29:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.958+0000 7fce926ca640 1 -- 172.21.15.64:0/3349906639 wait complete. 2023-11-24T06:29:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.960+0000 7fce926ca640 1 Processor -- start 2023-11-24T06:29:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.960+0000 7fce926ca640 1 -- start start 2023-11-24T06:29:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce926ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13cfa0 0x7fce8c13c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce926ca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce8c13fae0 0x7fce8c1f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce926ca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c15afb0 0x7fce8c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce926ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c157340 con 0x7fce8c13cfa0 2023-11-24T06:29:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce926ca640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1f6760 con 0x7fce8c15afb0 2023-11-24T06:29:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce926ca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1566c0 con 0x7fce8c13fae0 2023-11-24T06:29:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13cfa0 0x7fce8c13c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13cfa0 0x7fce8c13c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56370/0 (socket says 172.21.15.64:56370) 2023-11-24T06:29:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 -- 172.21.15.64:0/419168853 learned_addr learned my addr 172.21.15.64:0/419168853 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 -- 172.21.15.64:0/419168853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce8c13fae0 msgr2=0x7fce8c1f6220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:29:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce8c13fae0 0x7fce8c1f6220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 -- 172.21.15.64:0/419168853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c15afb0 msgr2=0x7fce8c0ca070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:29:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c15afb0 0x7fce8c0ca070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.961+0000 7fce916c8640 1 -- 172.21.15.64:0/419168853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce84009a70 con 0x7fce8c13cfa0 2023-11-24T06:29:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.962+0000 7fce916c8640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13cfa0 0x7fce8c13c7c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c700 tx=0x7fce8800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.962+0000 7fce7dffb640 1 -- 172.21.15.64:0/419168853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88005ce0 con 0x7fce8c13cfa0 2023-11-24T06:29:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.962+0000 7fce7dffb640 1 -- 172.21.15.64:0/419168853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce88005e40 con 0x7fce8c13cfa0 2023-11-24T06:29:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.962+0000 7fce7dffb640 1 -- 172.21.15.64:0/419168853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88007290 con 0x7fce8c13cfa0 2023-11-24T06:29:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.962+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c1569a0 con 0x7fce8c13cfa0 2023-11-24T06:29:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.962+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c156f60 con 0x7fce8c13cfa0 2023-11-24T06:29:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.963+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c067450 con 0x7fce8c13cfa0 2023-11-24T06:29:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.964+0000 7fce7dffb640 1 -- 172.21.15.64:0/419168853 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce880074b0 con 0x7fce8c13cfa0 2023-11-24T06:29:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.965+0000 7fce7dffb640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7006ae40 0x7fce7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.965+0000 7fce7dffb640 1 -- 172.21.15.64:0/419168853 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce88088a00 con 0x7fce8c13cfa0 2023-11-24T06:29:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.965+0000 7fce90ec7640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7006ae40 0x7fce7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.966+0000 7fce90ec7640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7006ae40 0x7fce7006d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce8c13cd00 tx=0x7fce80006b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:11.970+0000 7fce7dffb640 1 -- 172.21.15.64:0/419168853 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce88057530 con 0x7fce8c13cfa0 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.163+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce8c13ff30 con 0x7fce8c13cfa0 2023-11-24T06:29:12.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.166+0000 7fce7dffb640 1 -- 172.21.15.64:0/419168853 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16558 (secure 0 0 0) 0x7fce8c13ff30 con 0x7fce8c13cfa0 2023-11-24T06:29:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:12.169 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:10.723839+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7006ae40 msgr2=0x7fce7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7006ae40 0x7fce7006d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce8c13cd00 tx=0x7fce80006b80 comp rx=0 tx=0).stop 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13cfa0 msgr2=0x7fce8c13c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13cfa0 0x7fce8c13c7c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c700 tx=0x7fce8800cb00 comp rx=0 tx=0).stop 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 shutdown_connections 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7006ae40 0x7fce7006d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c15afb0 0x7fce8c0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce8c13fae0 0x7fce8c1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 --2- 172.21.15.64:0/419168853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce8c13cfa0 0x7fce8c13c7c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.168+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 >> 172.21.15.64:0/419168853 conn(0x7fce8c06a800 msgr2=0x7fce8c158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.169+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 shutdown_connections 2023-11-24T06:29:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.169+0000 7fce926ca640 1 -- 172.21.15.64:0/419168853 wait complete. 2023-11-24T06:29:12.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:29:12.211 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-11-24T06:29:12.209+0000 7f7f598e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:29:12.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.406+0000 7ff34aa2e640 1 Processor -- start 2023-11-24T06:29:12.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.406+0000 7ff34aa2e640 1 -- start start 2023-11-24T06:29:12.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff34aa2e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3440b35a0 0x7ff3440cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff34aa2e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3440ce440 con 0x7ff34413fcb0 2023-11-24T06:29:12.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff34aa2e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3440ce720 con 0x7ff34413cf10 2023-11-24T06:29:12.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff34aa2e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440cea00 con 0x7ff3440b35a0 2023-11-24T06:29:12.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff34922b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff34413fcb0 0x7ff3440cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33086/0 (socket says 172.21.15.64:33086) 2023-11-24T06:29:12.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff34922b640 1 -- 172.21.15.64:0/2878760247 learned_addr learned my addr 172.21.15.64:0/2878760247 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff34a22d640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3440b35a0 0x7ff3440cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.408+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760338415 0 0) 0x7ff3440ce440 con 0x7ff34413fcb0 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff330003520 con 0x7ff34413fcb0 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1971646391 0 0) 0x7ff330003520 con 0x7ff34413fcb0 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3440b35a0 msgr2=0x7ff3440cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3440b35a0 0x7ff3440cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 >> v1:172.21.15.162:6789/0 conn(0x7ff34413cf10 legacy=0x7ff34413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3440b1d90 con 0x7ff34413fcb0 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7ff334001670 con 0x7ff34413fcb0 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 >> v1:172.21.15.64:6789/0 conn(0x7ff34413fcb0 legacy=0x7ff3440cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff330003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3440b35a0 0x7ff3300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff330004d00 0x7ff330005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440cea00 con 0x7ff3300039a0 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33000ba60 con 0x7ff330004d00 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33000de30 con 0x7ff3440b35a0 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.409+0000 7ff349a2c640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff330003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.410+0000 7ff34a22d640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff330004d00 0x7ff330005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.410+0000 7ff349a2c640 1 -- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3440b35a0 msgr2=0x7ff3300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.410+0000 7ff349a2c640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3440b35a0 0x7ff3300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.410+0000 7ff349a2c640 1 -- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff330004d00 msgr2=0x7ff330005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.410+0000 7ff349a2c640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff330004d00 0x7ff330005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.410+0000 7ff349a2c640 1 -- 172.21.15.64:0/2878760247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330003520 con 0x7ff3300039a0 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.411+0000 7ff349a2c640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff330003dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff340001720 tx=0x7ff3300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba830975f7baea9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.411+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3400052d0 con 0x7ff3300039a0 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.411+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34002e050 con 0x7ff3300039a0 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.411+0000 7ff348a2a640 1 -- 172.21.15.64:0/2878760247 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340003830 con 0x7ff3300039a0 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.412+0000 7ff34aa2e640 1 -- 172.21.15.64:0/2878760247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 msgr2=0x7ff330003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.412+0000 7ff34aa2e640 1 --2- 172.21.15.64:0/2878760247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff330003dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff340001720 tx=0x7ff3300057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.412+0000 7ff34a22d640 1 -- 172.21.15.64:0/2878760247 reap_dead start 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.412+0000 7ff34aa2e640 1 -- 172.21.15.64:0/2878760247 shutdown_connections 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.412+0000 7ff34aa2e640 1 -- 172.21.15.64:0/2878760247 >> 172.21.15.64:0/2878760247 conn(0x7ff34406a7d0 msgr2=0x7ff34406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.412+0000 7ff34aa2e640 1 -- 172.21.15.64:0/2878760247 shutdown_connections 2023-11-24T06:29:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.412+0000 7ff34aa2e640 1 -- 172.21.15.64:0/2878760247 wait complete. 2023-11-24T06:29:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.414+0000 7ff34aa2e640 1 Processor -- start 2023-11-24T06:29:12.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.414+0000 7ff34aa2e640 1 -- start start 2023-11-24T06:29:12.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff34aa2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff34414c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff34aa2e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34415cf70 0x7ff34414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff34aa2e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34414a910 0x7ff34415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff34aa2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34414cde0 con 0x7ff3300039a0 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff34aa2e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34415c700 con 0x7ff34414a910 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff34aa2e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34414fc70 con 0x7ff34415cf70 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff349a2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff34414c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff349a2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff34414c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56384/0 (socket says 172.21.15.64:56384) 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.415+0000 7ff349a2c640 1 -- 172.21.15.64:0/860357579 learned_addr learned my addr 172.21.15.64:0/860357579 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff349a2c640 1 -- 172.21.15.64:0/860357579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34415cf70 msgr2=0x7ff34414a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff349a2c640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34415cf70 0x7ff34414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff349a2c640 1 -- 172.21.15.64:0/860357579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34414a910 msgr2=0x7ff34415bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff349a2c640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34414a910 0x7ff34415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff349a2c640 1 -- 172.21.15.64:0/860357579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34414ff50 con 0x7ff3300039a0 2023-11-24T06:29:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff349a2c640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff34414c8a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff340005040 tx=0x7ff340001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:12.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff33a7fc640 1 -- 172.21.15.64:0/860357579 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3400054b0 con 0x7ff3300039a0 2023-11-24T06:29:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff33a7fc640 1 -- 172.21.15.64:0/860357579 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff340007a30 con 0x7ff3300039a0 2023-11-24T06:29:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff33a7fc640 1 -- 172.21.15.64:0/860357579 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340006c20 con 0x7ff3300039a0 2023-11-24T06:29:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.416+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff344150230 con 0x7ff3300039a0 2023-11-24T06:29:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.417+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34414c030 con 0x7ff3300039a0 2023-11-24T06:29:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.418+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff344067450 con 0x7ff3300039a0 2023-11-24T06:29:12.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.419+0000 7ff33a7fc640 1 -- 172.21.15.64:0/860357579 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3400035f0 con 0x7ff3300039a0 2023-11-24T06:29:12.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.420+0000 7ff33a7fc640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff32806ae70 0x7ff32806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.420+0000 7ff33a7fc640 1 -- 172.21.15.64:0/860357579 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3400ab2a0 con 0x7ff3300039a0 2023-11-24T06:29:12.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.420+0000 7ff34922b640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff32806ae70 0x7ff32806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.421+0000 7ff34922b640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff32806ae70 0x7ff32806d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff34415b160 tx=0x7ff334001500 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:12.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.425+0000 7ff33a7fc640 1 -- 172.21.15.64:0/860357579 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3400762c0 con 0x7ff3300039a0 2023-11-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:12.660+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff34414cde0 con 0x7ff3300039a0 2023-11-24T06:29:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.293+0000 7ff33a7fc640 1 -- 172.21.15.64:0/860357579 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff3400798e0 con 0x7ff3300039a0 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff32806ae70 msgr2=0x7ff32806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff32806ae70 0x7ff32806d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff34415b160 tx=0x7ff334001500 comp rx=0 tx=0).stop 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 msgr2=0x7ff34414c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff34414c8a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff340005040 tx=0x7ff340001120 comp rx=0 tx=0).stop 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 shutdown_connections 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7ff32806ae70 0x7ff32806d330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34414a910 0x7ff34415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34415cf70 0x7ff34414a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 --2- 172.21.15.64:0/860357579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3300039a0 0x7ff34414c8a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 >> 172.21.15.64:0/860357579 conn(0x7ff34406a7d0 msgr2=0x7ff34413d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.297+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 shutdown_connections 2023-11-24T06:29:13.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.298+0000 7ff34aa2e640 1 -- 172.21.15.64:0/860357579 wait complete. 2023-11-24T06:29:13.324 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:29:13.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:29:13.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.601+0000 7f135867c640 1 Processor -- start 2023-11-24T06:29:13.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.601+0000 7f135867c640 1 -- start start 2023-11-24T06:29:13.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.602+0000 7f135867c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13500cd240 0x7f13500b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f135867c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f135009a170 0x7f13500cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f135867c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135006cea0 con 0x7f135006c6b0 2023-11-24T06:29:13.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f135867c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135006cbc0 con 0x7f13500cd240 2023-11-24T06:29:13.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f135867c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135006f350 con 0x7f135009a170 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f1356418640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13500cd240 0x7f13500b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f1356418640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13500cd240 0x7f13500b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:53720/0 (socket says 172.21.15.64:53720) 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f1356418640 1 -- 172.21.15.64:0/4202148470 learned_addr learned my addr 172.21.15.64:0/4202148470 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.603+0000 7f1356c19640 1 --2- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f135009a170 0x7f13500cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.604+0000 7f1356418640 1 -- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f135009a170 msgr2=0x7f13500cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.604+0000 7f1356418640 1 --2- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f135009a170 0x7f13500cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.604+0000 7f1356418640 1 -- 172.21.15.64:0/4202148470 >> v1:172.21.15.162:6789/0 conn(0x7f135006c6b0 legacy=0x7f1350099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.604+0000 7f1356418640 1 -- 172.21.15.64:0/4202148470 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135006f630 con 0x7f13500cd240 2023-11-24T06:29:13.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.605+0000 7f1356418640 1 --2- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13500cd240 0x7f13500b3820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f134c009e50 tx=0x7f134c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1939cab15d6dc756 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.606+0000 7f1355416640 1 -- 172.21.15.64:0/4202148470 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c02cb00 con 0x7f13500cd240 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.606+0000 7f1355416640 1 -- 172.21.15.64:0/4202148470 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c02cc60 con 0x7f13500cd240 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.606+0000 7f1355416640 1 -- 172.21.15.64:0/4202148470 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0343f0 con 0x7f13500cd240 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 -- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13500cd240 msgr2=0x7f13500b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 --2- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13500cd240 0x7f13500b3820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f134c009e50 tx=0x7f134c02bf60 comp rx=0 tx=0).stop 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 -- 172.21.15.64:0/4202148470 shutdown_connections 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 --2- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f135009a170 0x7f13500cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 --2- 172.21.15.64:0/4202148470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13500cd240 0x7f13500b3820 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 -- 172.21.15.64:0/4202148470 >> 172.21.15.64:0/4202148470 conn(0x7f13500b2d00 msgr2=0x7f13500b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 -- 172.21.15.64:0/4202148470 shutdown_connections 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.607+0000 7f135867c640 1 -- 172.21.15.64:0/4202148470 wait complete. 2023-11-24T06:29:13.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.609+0000 7f135867c640 1 Processor -- start 2023-11-24T06:29:13.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.609+0000 7f135867c640 1 -- start start 2023-11-24T06:29:13.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.610+0000 7f135867c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f135006c6b0 0x7f13500ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.610+0000 7f135867c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135009a170 0x7f135015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.610+0000 7f135867c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13500cd240 0x7f13501f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.610+0000 7f135867c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501f3250 con 0x7f135009a170 2023-11-24T06:29:13.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.610+0000 7f135867c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13500ca510 con 0x7f135006c6b0 2023-11-24T06:29:13.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.610+0000 7f135867c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13500cade0 con 0x7f13500cd240 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135009a170 0x7f135015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135009a170 0x7f135015a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37288/0 (socket says 172.21.15.64:37288) 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 -- 172.21.15.64:0/634658193 learned_addr learned my addr 172.21.15.64:0/634658193 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1356c19640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13500cd240 0x7f13501f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1356418640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f135006c6b0 0x7f13500ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 -- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13500cd240 msgr2=0x7f13501f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13500cd240 0x7f13501f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 -- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f135006c6b0 msgr2=0x7f13500ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f135006c6b0 0x7f13500ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1355c17640 1 -- 172.21.15.64:0/634658193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c009b30 con 0x7f135009a170 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.611+0000 7f1356418640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f135006c6b0 0x7f13500ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.612+0000 7f1355c17640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135009a170 0x7f135015a740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f134000c440 tx=0x7f134000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.612+0000 7f1346ffd640 1 -- 172.21.15.64:0/634658193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400052d0 con 0x7f135009a170 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.612+0000 7f1346ffd640 1 -- 172.21.15.64:0/634658193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1340005430 con 0x7f135009a170 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.612+0000 7f1346ffd640 1 -- 172.21.15.64:0/634658193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340007260 con 0x7f135009a170 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.612+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13500c87c0 con 0x7f135009a170 2023-11-24T06:29:13.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.612+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13500c8d80 con 0x7f135009a170 2023-11-24T06:29:13.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.613+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13501f33c0 con 0x7f135009a170 2023-11-24T06:29:13.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.614+0000 7f1346ffd640 1 -- 172.21.15.64:0/634658193 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13400073c0 con 0x7f135009a170 2023-11-24T06:29:13.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.615+0000 7f1346ffd640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f133406ae70 0x7f133406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.615+0000 7f1346ffd640 1 -- 172.21.15.64:0/634658193 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1340088940 con 0x7f135009a170 2023-11-24T06:29:13.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.616+0000 7f1356418640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f133406ae70 0x7f133406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.616+0000 7f1356418640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f133406ae70 0x7f133406d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f134c035000 tx=0x7f134c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.620+0000 7f1346ffd640 1 -- 172.21.15.64:0/634658193 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13400539f0 con 0x7f135009a170 2023-11-24T06:29:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:13.854+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f13500675d0 con 0x7f135009a170 2023-11-24T06:29:14.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.297+0000 7f1346ffd640 1 -- 172.21.15.64:0/634658193 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1340057010 con 0x7f135009a170 2023-11-24T06:29:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f133406ae70 msgr2=0x7f133406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f133406ae70 0x7f133406d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f134c035000 tx=0x7f134c030000 comp rx=0 tx=0).stop 2023-11-24T06:29:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135009a170 msgr2=0x7f135015a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135009a170 0x7f135015a740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f134000c440 tx=0x7f134000c900 comp rx=0 tx=0).stop 2023-11-24T06:29:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 shutdown_connections 2023-11-24T06:29:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f133406ae70 0x7f133406d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13500cd240 0x7f13501f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f135009a170 0x7f135015a740 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 --2- 172.21.15.64:0/634658193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f135006c6b0 0x7f13500ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 >> 172.21.15.64:0/634658193 conn(0x7f13500b2d00 msgr2=0x7f1350099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 shutdown_connections 2023-11-24T06:29:14.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.301+0000 7f135867c640 1 -- 172.21.15.64:0/634658193 wait complete. 2023-11-24T06:29:14.323 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:29:14.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:14.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.548+0000 7f7bef8ad640 1 Processor -- start 2023-11-24T06:29:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.548+0000 7f7bef8ad640 1 -- start start 2023-11-24T06:29:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.549+0000 7f7bef8ad640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7be80cf9a0 0x7f7be8166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bef8ad640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be815c5f0 con 0x7f7be813df80 2023-11-24T06:29:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bef8ad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be806c470 con 0x7f7be813e360 2023-11-24T06:29:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bef8ad640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be806c750 con 0x7f7be80cf9a0 2023-11-24T06:29:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bed649640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7be813df80 0x7f7be815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59518/0 (socket says 172.21.15.64:59518) 2023-11-24T06:29:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bed649640 1 -- 172.21.15.64:0/3331740613 learned_addr learned my addr 172.21.15.64:0/3331740613 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bede4a640 1 --2- 172.21.15.64:0/3331740613 >> v2:172.21.15.64:3300/0 conn(0x7f7be80cf9a0 0x7f7be8166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bede4a640 1 -- 172.21.15.64:0/3331740613 >> v1:172.21.15.162:6789/0 conn(0x7f7be813e360 legacy=0x7f7be80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bede4a640 1 -- 172.21.15.64:0/3331740613 >> v1:172.21.15.64:6789/0 conn(0x7f7be813df80 legacy=0x7f7be815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bede4a640 1 -- 172.21.15.64:0/3331740613 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be806ca30 con 0x7f7be80cf9a0 2023-11-24T06:29:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.550+0000 7f7bdffff640 1 -- 172.21.15.64:0/3331740613 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916278937 0 0) 0x7f7be815c5f0 con 0x7f7be813df80 2023-11-24T06:29:14.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.552+0000 7f7bede4a640 1 --2- 172.21.15.64:0/3331740613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be80cf9a0 0x7f7be8166030 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827abecc071fdc5a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:14.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.552+0000 7f7bdffff640 1 -- 172.21.15.64:0/3331740613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0031830 con 0x7f7be80cf9a0 2023-11-24T06:29:14.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.552+0000 7f7bdffff640 1 -- 172.21.15.64:0/3331740613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be0031990 con 0x7f7be80cf9a0 2023-11-24T06:29:14.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.552+0000 7f7bdffff640 1 -- 172.21.15.64:0/3331740613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0031d20 con 0x7f7be80cf9a0 2023-11-24T06:29:14.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.552+0000 7f7bef8ad640 1 -- 172.21.15.64:0/3331740613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be80cf9a0 msgr2=0x7f7be8166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.552+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/3331740613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be80cf9a0 0x7f7be8166030 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.553+0000 7f7bef8ad640 1 -- 172.21.15.64:0/3331740613 shutdown_connections 2023-11-24T06:29:14.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.553+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/3331740613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be80cf9a0 0x7f7be8166030 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.553+0000 7f7bef8ad640 1 -- 172.21.15.64:0/3331740613 >> 172.21.15.64:0/3331740613 conn(0x7f7be806a800 msgr2=0x7f7be806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:14.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.553+0000 7f7bef8ad640 1 -- 172.21.15.64:0/3331740613 shutdown_connections 2023-11-24T06:29:14.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.553+0000 7f7bef8ad640 1 -- 172.21.15.64:0/3331740613 wait complete. 2023-11-24T06:29:14.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.554+0000 7f7bef8ad640 1 Processor -- start 2023-11-24T06:29:14.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.555+0000 7f7bef8ad640 1 -- start start 2023-11-24T06:29:14.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.555+0000 7f7bef8ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7be80cf9a0 0x7f7be81f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:14.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.555+0000 7f7bef8ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be813df80 0x7f7be8070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.555+0000 7f7bef8ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7be813e360 0x7f7be8070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.555+0000 7f7bef8ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be814ad30 con 0x7f7be813df80 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.555+0000 7f7bef8ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be8071400 con 0x7f7be80cf9a0 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.555+0000 7f7bef8ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be80716e0 con 0x7f7be813e360 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be813df80 0x7f7be8070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be813df80 0x7f7be8070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37304/0 (socket says 172.21.15.64:37304) 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 -- 172.21.15.64:0/2815995175 learned_addr learned my addr 172.21.15.64:0/2815995175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bed649640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7be80cf9a0 0x7f7be81f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:14.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bede4a640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7be813e360 0x7f7be8070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:14.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 -- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7be813e360 msgr2=0x7f7be8070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7be813e360 0x7f7be8070dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 -- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7be80cf9a0 msgr2=0x7f7be81f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7be80cf9a0 0x7f7be81f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.556+0000 7f7bece48640 1 -- 172.21.15.64:0/2815995175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009400 con 0x7f7be813df80 2023-11-24T06:29:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.557+0000 7f7bece48640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be813df80 0x7f7be8070890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800c7f0 tx=0x7f7bd800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.557+0000 7f7bddffb640 1 -- 172.21.15.64:0/2815995175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8003d80 con 0x7f7be813df80 2023-11-24T06:29:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.557+0000 7f7bddffb640 1 -- 172.21.15.64:0/2815995175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd8005070 con 0x7f7be813df80 2023-11-24T06:29:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.557+0000 7f7bddffb640 1 -- 172.21.15.64:0/2815995175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8005460 con 0x7f7be813df80 2023-11-24T06:29:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.557+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be80719c0 con 0x7f7be813df80 2023-11-24T06:29:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.557+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be8071f80 con 0x7f7be813df80 2023-11-24T06:29:14.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.558+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be80721a0 con 0x7f7be813df80 2023-11-24T06:29:14.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.559+0000 7f7bddffb640 1 -- 172.21.15.64:0/2815995175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bd80055c0 con 0x7f7be813df80 2023-11-24T06:29:14.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.560+0000 7f7bddffb640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7bcc06ae40 0x7f7bcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:14.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.561+0000 7f7bddffb640 1 -- 172.21.15.64:0/2815995175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bd8088c10 con 0x7f7be813df80 2023-11-24T06:29:14.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.561+0000 7f7bed649640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7bcc06ae40 0x7f7bcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:14.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.562+0000 7f7bed649640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7bcc06ae40 0x7f7bcc06d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000f60 tx=0x7f7be4007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:14.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.565+0000 7f7bddffb640 1 -- 172.21.15.64:0/2815995175 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bd8053cb0 con 0x7f7be813df80 2023-11-24T06:29:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.794+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7be80675d0 con 0x7f7be813df80 2023-11-24T06:29:14.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.795+0000 7f7bddffb640 1 -- 172.21.15.64:0/2815995175 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f7bd8017020 con 0x7f7be813df80 2023-11-24T06:29:14.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:14.799 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3831306721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3831306721},{"type":"v1","addr":"172.21.15.162:6834","nonce":3831306721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1401622125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1401622125},{"type":"v1","addr":"172.21.15.162:6835","nonce":1401622125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2378368513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6835","nonce":2378368513},{"type":"v1","addr":"172.21.15.64:6837","nonce":2378368513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:10.724367+0000","modified":"2023-11-24T06:29:13.855414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6836/2541249514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2541249514},{"type":"v1","addr":"172.21.15.64:6836","nonce":2541249514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:29:14.800 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7bcc06ae40 msgr2=0x7f7bcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7bcc06ae40 0x7f7bcc06d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000f60 tx=0x7f7be4007c00 comp rx=0 tx=0).stop 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be813df80 msgr2=0x7f7be8070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be813df80 0x7f7be8070890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800c7f0 tx=0x7f7bd800ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 shutdown_connections 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7bcc06ae40 0x7f7bcc06d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7be813e360 0x7f7be8070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7be813df80 0x7f7be8070890 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 --2- 172.21.15.64:0/2815995175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7be80cf9a0 0x7f7be81f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:14.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 >> 172.21.15.64:0/2815995175 conn(0x7f7be806a800 msgr2=0x7f7be80cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:14.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 shutdown_connections 2023-11-24T06:29:14.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:14.798+0000 7f7bef8ad640 1 -- 172.21.15.64:0/2815995175 wait complete. 2023-11-24T06:29:14.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.037+0000 7f41c2bfa640 1 Processor -- start 2023-11-24T06:29:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.038+0000 7f41c2bfa640 1 -- start start 2023-11-24T06:29:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.039+0000 7f41c2bfa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 0x7f41bc0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.039+0000 7f41c2bfa640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41bc06d160 0x7f41bc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c2bfa640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc070e50 con 0x7f41bc143c40 2023-11-24T06:29:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c2bfa640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc0723c0 con 0x7f41bc06d160 2023-11-24T06:29:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c2bfa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc144020 con 0x7f41bc0cc500 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c23f9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41bc06d160 0x7f41bc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c23f9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41bc06d160 0x7f41bc06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37306/0 (socket says 172.21.15.64:37306) 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c23f9640 1 -- 172.21.15.64:0/1144044047 learned_addr learned my addr 172.21.15.64:0/1144044047 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c13f7640 1 --2- 172.21.15.64:0/1144044047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 0x7f41bc0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.040+0000 7f41c23f9640 1 -- 172.21.15.64:0/1144044047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 msgr2=0x7f41bc0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.041+0000 7f41c23f9640 1 --2- 172.21.15.64:0/1144044047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 0x7f41bc0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.041+0000 7f41c23f9640 1 -- 172.21.15.64:0/1144044047 >> v1:172.21.15.162:6789/0 conn(0x7f41bc143c40 legacy=0x7f41bc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.041+0000 7f41c23f9640 1 -- 172.21.15.64:0/1144044047 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0ccea0 con 0x7f41bc06d160 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.042+0000 7f41c23f9640 1 --2- 172.21.15.64:0/1144044047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc06d590 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009e50 tx=0x7f41b402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7a5a9997bcc672 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.042+0000 7f41c0bf6640 1 -- 172.21.15.64:0/1144044047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b402c9e0 con 0x7f41bc06d160 2023-11-24T06:29:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.042+0000 7f41c0bf6640 1 -- 172.21.15.64:0/1144044047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b402cb40 con 0x7f41bc06d160 2023-11-24T06:29:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.042+0000 7f41c0bf6640 1 -- 172.21.15.64:0/1144044047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b40343f0 con 0x7f41bc06d160 2023-11-24T06:29:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 -- 172.21.15.64:0/1144044047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 msgr2=0x7f41bc06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/1144044047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc06d590 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009e50 tx=0x7f41b402be40 comp rx=0 tx=0).stop 2023-11-24T06:29:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 -- 172.21.15.64:0/1144044047 shutdown_connections 2023-11-24T06:29:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/1144044047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc06d590 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/1144044047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 0x7f41bc0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 -- 172.21.15.64:0/1144044047 >> 172.21.15.64:0/1144044047 conn(0x7f41bc06a800 msgr2=0x7f41bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 -- 172.21.15.64:0/1144044047 shutdown_connections 2023-11-24T06:29:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.043+0000 7f41c2bfa640 1 -- 172.21.15.64:0/1144044047 wait complete. 2023-11-24T06:29:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.045+0000 7f41c2bfa640 1 Processor -- start 2023-11-24T06:29:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.045+0000 7f41c2bfa640 1 -- start start 2023-11-24T06:29:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c2bfa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c2bfa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 0x7f41bc13a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c2bfa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41bc143c40 0x7f41bc131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c2bfa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc13a060 con 0x7f41bc06d160 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c2bfa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc1391d0 con 0x7f41bc143c40 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c2bfa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc15af30 con 0x7f41bc0cc500 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37322/0 (socket says 172.21.15.64:37322) 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 -- 172.21.15.64:0/714024164 learned_addr learned my addr 172.21.15.64:0/714024164 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 -- 172.21.15.64:0/714024164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 msgr2=0x7f41bc13a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 0x7f41bc13a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 -- 172.21.15.64:0/714024164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41bc143c40 msgr2=0x7f41bc131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41bc143c40 0x7f41bc131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.046+0000 7f41c1bf8640 1 -- 172.21.15.64:0/714024164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4009b30 con 0x7f41bc06d160 2023-11-24T06:29:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.047+0000 7f41c1bf8640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc15a9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f41b800b3f0 tx=0x7f41b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.047+0000 7f41b27fc640 1 -- 172.21.15.64:0/714024164 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8011830 con 0x7f41bc06d160 2023-11-24T06:29:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.047+0000 7f41b27fc640 1 -- 172.21.15.64:0/714024164 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b8011990 con 0x7f41bc06d160 2023-11-24T06:29:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.047+0000 7f41b27fc640 1 -- 172.21.15.64:0/714024164 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8011d40 con 0x7f41bc06d160 2023-11-24T06:29:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.047+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc1393e0 con 0x7f41bc06d160 2023-11-24T06:29:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.047+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1399a0 con 0x7f41bc06d160 2023-11-24T06:29:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.048+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc13a2d0 con 0x7f41bc06d160 2023-11-24T06:29:15.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.049+0000 7f41b27fc640 1 -- 172.21.15.64:0/714024164 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41b801a400 con 0x7f41bc06d160 2023-11-24T06:29:15.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.050+0000 7f41b27fc640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a006ae40 0x7f41a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.050+0000 7f41b27fc640 1 -- 172.21.15.64:0/714024164 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41b8089440 con 0x7f41bc06d160 2023-11-24T06:29:15.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.050+0000 7f41c13f7640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a006ae40 0x7f41a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.051+0000 7f41c13f7640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a006ae40 0x7f41a006d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f41bc13da50 tx=0x7f41ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:15.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.054+0000 7f41b27fc640 1 -- 172.21.15.64:0/714024164 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41b80544f0 con 0x7f41bc06d160 2023-11-24T06:29:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.246+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41bc067450 con 0x7f41bc06d160 2023-11-24T06:29:15.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.248+0000 7f41b27fc640 1 -- 172.21.15.64:0/714024164 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f41b8057b10 con 0x7f41bc06d160 2023-11-24T06:29:15.251 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:12.271627+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.250+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a006ae40 msgr2=0x7f41a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.250+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a006ae40 0x7f41a006d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f41bc13da50 tx=0x7f41ac008040 comp rx=0 tx=0).stop 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.250+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 msgr2=0x7f41bc15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.250+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc15a9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f41b800b3f0 tx=0x7f41b800b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 shutdown_connections 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f41a006ae40 0x7f41a006d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41bc143c40 0x7f41bc131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41bc0cc500 0x7f41bc13a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 --2- 172.21.15.64:0/714024164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41bc06d160 0x7f41bc15a9f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 >> 172.21.15.64:0/714024164 conn(0x7f41bc06a800 msgr2=0x7f41bc071290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 shutdown_connections 2023-11-24T06:29:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.251+0000 7f41c2bfa640 1 -- 172.21.15.64:0/714024164 wait complete. 2023-11-24T06:29:15.272 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:29:15.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:15.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.487+0000 7f50527a5640 1 Processor -- start 2023-11-24T06:29:15.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.487+0000 7f50527a5640 1 -- start start 2023-11-24T06:29:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.489+0000 7f50527a5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f504c143c40 0x7f504c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.489+0000 7f50527a5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f504c0717d0 0x7f504c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.489+0000 7f50527a5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c070e50 con 0x7f504c0cca40 2023-11-24T06:29:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.489+0000 7f50527a5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0723c0 con 0x7f504c0717d0 2023-11-24T06:29:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.489+0000 7f50527a5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0cce40 con 0x7f504c143c40 2023-11-24T06:29:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.489+0000 7f5050fa2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f504c0717d0 0x7f504c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.490+0000 7f5050fa2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f504c0717d0 0x7f504c0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37330/0 (socket says 172.21.15.64:37330) 2023-11-24T06:29:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.490+0000 7f5050fa2640 1 -- 172.21.15.64:0/2437092531 learned_addr learned my addr 172.21.15.64:0/2437092531 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.490+0000 7f50517a3640 1 --2- 172.21.15.64:0/2437092531 >> v2:172.21.15.162:3300/0 conn(0x7f504c143c40 0x7f504c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.490+0000 7f5050fa2640 1 -- 172.21.15.64:0/2437092531 >> v2:172.21.15.162:3300/0 conn(0x7f504c143c40 msgr2=0x7f504c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.490+0000 7f5050fa2640 1 --2- 172.21.15.64:0/2437092531 >> v2:172.21.15.162:3300/0 conn(0x7f504c143c40 0x7f504c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.490+0000 7f5050fa2640 1 -- 172.21.15.64:0/2437092531 >> v1:172.21.15.162:6789/0 conn(0x7f504c0cca40 legacy=0x7f504c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:15.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.490+0000 7f5050fa2640 1 -- 172.21.15.64:0/2437092531 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c06e170 con 0x7f504c0717d0 2023-11-24T06:29:15.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.491+0000 7f5050fa2640 1 --2- 172.21.15.64:0/2437092531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0717d0 0x7f504c0cc500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f504c06e410 tx=0x7f503c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba00e07837215851 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:15.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.491+0000 7f5043fff640 1 -- 172.21.15.64:0/2437092531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c0317e0 con 0x7f504c0717d0 2023-11-24T06:29:15.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f5043fff640 1 -- 172.21.15.64:0/2437092531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c031940 con 0x7f504c0717d0 2023-11-24T06:29:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f5043fff640 1 -- 172.21.15.64:0/2437092531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c031cd0 con 0x7f504c0717d0 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f50527a5640 1 -- 172.21.15.64:0/2437092531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0717d0 msgr2=0x7f504c0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f50527a5640 1 --2- 172.21.15.64:0/2437092531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0717d0 0x7f504c0cc500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f504c06e410 tx=0x7f503c02b880 comp rx=0 tx=0).stop 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f50527a5640 1 -- 172.21.15.64:0/2437092531 shutdown_connections 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f50527a5640 1 --2- 172.21.15.64:0/2437092531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0717d0 0x7f504c0cc500 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f50527a5640 1 --2- 172.21.15.64:0/2437092531 >> v2:172.21.15.162:3300/0 conn(0x7f504c143c40 0x7f504c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.492+0000 7f50527a5640 1 -- 172.21.15.64:0/2437092531 >> 172.21.15.64:0/2437092531 conn(0x7f504c06a800 msgr2=0x7f504c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.493+0000 7f50527a5640 1 -- 172.21.15.64:0/2437092531 shutdown_connections 2023-11-24T06:29:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.493+0000 7f50527a5640 1 -- 172.21.15.64:0/2437092531 wait complete. 2023-11-24T06:29:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.494+0000 7f50527a5640 1 Processor -- start 2023-11-24T06:29:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.494+0000 7f50527a5640 1 -- start start 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.495+0000 7f50527a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c0717d0 0x7f504c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.495+0000 7f50527a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0cca40 0x7f504c136b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.495+0000 7f50527a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c143c40 0x7f504c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.495+0000 7f50527a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0b2f60 con 0x7f504c0cca40 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.495+0000 7f50527a5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0b20d0 con 0x7f504c0717d0 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.495+0000 7f50527a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0b22e0 con 0x7f504c143c40 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0cca40 0x7f504c136b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0cca40 0x7f504c136b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37342/0 (socket says 172.21.15.64:37342) 2023-11-24T06:29:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 -- 172.21.15.64:0/1752618860 learned_addr learned my addr 172.21.15.64:0/1752618860 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 -- 172.21.15.64:0/1752618860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c143c40 msgr2=0x7f504c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c143c40 0x7f504c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 -- 172.21.15.64:0/1752618860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c0717d0 msgr2=0x7f504c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c0717d0 0x7f504c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.496+0000 7f5050fa2640 1 -- 172.21.15.64:0/1752618860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0094d0 con 0x7f504c0cca40 2023-11-24T06:29:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.497+0000 7f5050fa2640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0cca40 0x7f504c136b50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f503c02b8c0 tx=0x7f503c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.497+0000 7f5041ffb640 1 -- 172.21.15.64:0/1752618860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c032880 con 0x7f504c0cca40 2023-11-24T06:29:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.497+0000 7f5041ffb640 1 -- 172.21.15.64:0/1752618860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c0329e0 con 0x7f504c0cca40 2023-11-24T06:29:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.497+0000 7f5041ffb640 1 -- 172.21.15.64:0/1752618860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c032d90 con 0x7f504c0cca40 2023-11-24T06:29:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.497+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c0b25c0 con 0x7f504c0cca40 2023-11-24T06:29:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.497+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c0b2b80 con 0x7f504c0cca40 2023-11-24T06:29:15.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.499+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c067450 con 0x7f504c0cca40 2023-11-24T06:29:15.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.499+0000 7f5041ffb640 1 -- 172.21.15.64:0/1752618860 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f503c04e450 con 0x7f504c0cca40 2023-11-24T06:29:15.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.500+0000 7f5041ffb640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f503006ae40 0x7f503006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:15.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.500+0000 7f5041ffb640 1 -- 172.21.15.64:0/1752618860 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f503c0ab710 con 0x7f504c0cca40 2023-11-24T06:29:15.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.501+0000 7f50517a3640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f503006ae40 0x7f503006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.501+0000 7f50517a3640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f503006ae40 0x7f503006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5048006fd0 tx=0x7f5048008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:15.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.505+0000 7f5041ffb640 1 -- 172.21.15.64:0/1752618860 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f503c076870 con 0x7f504c0cca40 2023-11-24T06:29:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.691+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f504c0720e0 con 0x7f503006ae40 2023-11-24T06:29:15.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.720+0000 7f5041ffb640 1 -- 172.21.15.64:0/1752618860 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286313 (secure 0 0 0) 0x7f504800be20 con 0x7f503006ae40 2023-11-24T06:29:15.726 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:15.728 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.726+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f503006ae40 msgr2=0x7f503006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.726+0000 7f50527a5640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f503006ae40 0x7f503006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5048006fd0 tx=0x7f5048008040 comp rx=0 tx=0).stop 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0cca40 msgr2=0x7f504c136b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0cca40 0x7f504c136b50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f503c02b8c0 tx=0x7f503c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 shutdown_connections 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f503006ae40 0x7f503006d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c143c40 0x7f504c0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f504c0cca40 0x7f504c136b50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 --2- 172.21.15.64:0/1752618860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c0717d0 0x7f504c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 >> 172.21.15.64:0/1752618860 conn(0x7f504c06a800 msgr2=0x7f504c13f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 shutdown_connections 2023-11-24T06:29:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:15.727+0000 7f50527a5640 1 -- 172.21.15.64:0/1752618860 wait complete. 2023-11-24T06:29:15.747 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-24T06:29:15.128247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220564,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754156,"statfs":{"total":773094113280,"available":772868255744,"internally_reserved":0,"allocated":4497408,"data_stored":2462844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.897278"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681781+0000","last_change":"2023-11-24T06:29:11.282621+0000","last_active":"2023-11-24T06:29:12.681781+0000","last_peered":"2023-11-24T06:29:12.681781+0000","last_clean":"2023-11-24T06:29:12.681781+0000","last_became_active":"2023-11-24T06:29:11.282310+0000","last_became_peered":"2023-11-24T06:29:11.282310+0000","last_unstale":"2023-11-24T06:29:12.681781+0000","last_undegraded":"2023-11-24T06:29:12.681781+0000","last_fullsized":"2023-11-24T06:29:12.681781+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:47.628375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683470+0000","last_change":"2023-11-24T06:29:11.311148+0000","last_active":"2023-11-24T06:29:12.683470+0000","last_peered":"2023-11-24T06:29:12.683470+0000","last_clean":"2023-11-24T06:29:12.683470+0000","last_became_active":"2023-11-24T06:29:11.308037+0000","last_became_peered":"2023-11-24T06:29:11.308037+0000","last_unstale":"2023-11-24T06:29:12.683470+0000","last_undegraded":"2023-11-24T06:29:12.683470+0000","last_fullsized":"2023-11-24T06:29:12.683470+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:02.393482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290475+0000","last_change":"2023-11-24T06:29:11.293230+0000","last_active":"2023-11-24T06:29:12.290475+0000","last_peered":"2023-11-24T06:29:12.290475+0000","last_clean":"2023-11-24T06:29:12.290475+0000","last_became_active":"2023-11-24T06:29:11.292929+0000","last_became_peered":"2023-11-24T06:29:11.292929+0000","last_unstale":"2023-11-24T06:29:12.290475+0000","last_undegraded":"2023-11-24T06:29:12.290475+0000","last_fullsized":"2023-11-24T06:29:12.290475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:24.991791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681028+0000","last_change":"2023-11-24T06:29:11.287940+0000","last_active":"2023-11-24T06:29:12.681028+0000","last_peered":"2023-11-24T06:29:12.681028+0000","last_clean":"2023-11-24T06:29:12.681028+0000","last_became_active":"2023-11-24T06:29:11.287627+0000","last_became_peered":"2023-11-24T06:29:11.287627+0000","last_unstale":"2023-11-24T06:29:12.681028+0000","last_undegraded":"2023-11-24T06:29:12.681028+0000","last_fullsized":"2023-11-24T06:29:12.681028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:15.117892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285485+0000","last_change":"2023-11-24T06:29:11.309984+0000","last_active":"2023-11-24T06:29:12.285485+0000","last_peered":"2023-11-24T06:29:12.285485+0000","last_clean":"2023-11-24T06:29:12.285485+0000","last_became_active":"2023-11-24T06:29:11.309562+0000","last_became_peered":"2023-11-24T06:29:11.309562+0000","last_unstale":"2023-11-24T06:29:12.285485+0000","last_undegraded":"2023-11-24T06:29:12.285485+0000","last_fullsized":"2023-11-24T06:29:12.285485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:57.411653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681528+0000","last_change":"2023-11-24T06:29:11.308671+0000","last_active":"2023-11-24T06:29:12.681528+0000","last_peered":"2023-11-24T06:29:12.681528+0000","last_clean":"2023-11-24T06:29:12.681528+0000","last_became_active":"2023-11-24T06:29:11.308342+0000","last_became_peered":"2023-11-24T06:29:11.308342+0000","last_unstale":"2023-11-24T06:29:12.681528+0000","last_undegraded":"2023-11-24T06:29:12.681528+0000","last_fullsized":"2023-11-24T06:29:12.681528+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:56.242991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328062+0000","last_change":"2023-11-24T06:29:11.287205+0000","last_active":"2023-11-24T06:29:12.328062+0000","last_peered":"2023-11-24T06:29:12.328062+0000","last_clean":"2023-11-24T06:29:12.328062+0000","last_became_active":"2023-11-24T06:29:11.285679+0000","last_became_peered":"2023-11-24T06:29:11.285679+0000","last_unstale":"2023-11-24T06:29:12.328062+0000","last_undegraded":"2023-11-24T06:29:12.328062+0000","last_fullsized":"2023-11-24T06:29:12.328062+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:30.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327290+0000","last_change":"2023-11-24T06:29:11.276395+0000","last_active":"2023-11-24T06:29:12.327290+0000","last_peered":"2023-11-24T06:29:12.327290+0000","last_clean":"2023-11-24T06:29:12.327290+0000","last_became_active":"2023-11-24T06:29:11.275962+0000","last_became_peered":"2023-11-24T06:29:11.275962+0000","last_unstale":"2023-11-24T06:29:12.327290+0000","last_undegraded":"2023-11-24T06:29:12.327290+0000","last_fullsized":"2023-11-24T06:29:12.327290+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:36.625390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289719+0000","last_change":"2023-11-24T06:29:11.281288+0000","last_active":"2023-11-24T06:29:12.289719+0000","last_peered":"2023-11-24T06:29:12.289719+0000","last_clean":"2023-11-24T06:29:12.289719+0000","last_became_active":"2023-11-24T06:29:11.280975+0000","last_became_peered":"2023-11-24T06:29:11.280975+0000","last_unstale":"2023-11-24T06:29:12.289719+0000","last_undegraded":"2023-11-24T06:29:12.289719+0000","last_fullsized":"2023-11-24T06:29:12.289719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:24.991791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681158+0000","last_change":"2023-11-24T06:29:11.311472+0000","last_active":"2023-11-24T06:29:12.681158+0000","last_peered":"2023-11-24T06:29:12.681158+0000","last_clean":"2023-11-24T06:29:12.681158+0000","last_became_active":"2023-11-24T06:29:11.311169+0000","last_became_peered":"2023-11-24T06:29:11.311169+0000","last_unstale":"2023-11-24T06:29:12.681158+0000","last_undegraded":"2023-11-24T06:29:12.681158+0000","last_fullsized":"2023-11-24T06:29:12.681158+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:15:34.296527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328282+0000","last_change":"2023-11-24T06:29:11.288305+0000","last_active":"2023-11-24T06:29:12.328282+0000","last_peered":"2023-11-24T06:29:12.328282+0000","last_clean":"2023-11-24T06:29:12.328282+0000","last_became_active":"2023-11-24T06:29:11.288019+0000","last_became_peered":"2023-11-24T06:29:11.288019+0000","last_unstale":"2023-11-24T06:29:12.328282+0000","last_undegraded":"2023-11-24T06:29:12.328282+0000","last_fullsized":"2023-11-24T06:29:12.328282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:48.087034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290656+0000","last_change":"2023-11-24T06:29:11.311271+0000","last_active":"2023-11-24T06:29:12.290656+0000","last_peered":"2023-11-24T06:29:12.290656+0000","last_clean":"2023-11-24T06:29:12.290656+0000","last_became_active":"2023-11-24T06:29:11.310087+0000","last_became_peered":"2023-11-24T06:29:11.310087+0000","last_unstale":"2023-11-24T06:29:12.290656+0000","last_undegraded":"2023-11-24T06:29:12.290656+0000","last_fullsized":"2023-11-24T06:29:12.290656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:18.092054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286020+0000","last_change":"2023-11-24T06:29:11.282400+0000","last_active":"2023-11-24T06:29:12.286020+0000","last_peered":"2023-11-24T06:29:12.286020+0000","last_clean":"2023-11-24T06:29:12.286020+0000","last_became_active":"2023-11-24T06:29:11.280456+0000","last_became_peered":"2023-11-24T06:29:11.280456+0000","last_unstale":"2023-11-24T06:29:12.286020+0000","last_undegraded":"2023-11-24T06:29:12.286020+0000","last_fullsized":"2023-11-24T06:29:12.286020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:07:23.526675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681698+0000","last_change":"2023-11-24T06:29:11.310568+0000","last_active":"2023-11-24T06:29:12.681698+0000","last_peered":"2023-11-24T06:29:12.681698+0000","last_clean":"2023-11-24T06:29:12.681698+0000","last_became_active":"2023-11-24T06:29:11.307987+0000","last_became_peered":"2023-11-24T06:29:11.307987+0000","last_unstale":"2023-11-24T06:29:12.681698+0000","last_undegraded":"2023-11-24T06:29:12.681698+0000","last_fullsized":"2023-11-24T06:29:12.681698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:00.524180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289558+0000","last_change":"2023-11-24T06:29:11.288420+0000","last_active":"2023-11-24T06:29:12.289558+0000","last_peered":"2023-11-24T06:29:12.289558+0000","last_clean":"2023-11-24T06:29:12.289558+0000","last_became_active":"2023-11-24T06:29:11.286993+0000","last_became_peered":"2023-11-24T06:29:11.286993+0000","last_unstale":"2023-11-24T06:29:12.289558+0000","last_undegraded":"2023-11-24T06:29:12.289558+0000","last_fullsized":"2023-11-24T06:29:12.289558+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:29.315454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328224+0000","last_change":"2023-11-24T06:29:11.314102+0000","last_active":"2023-11-24T06:29:12.328224+0000","last_peered":"2023-11-24T06:29:12.328224+0000","last_clean":"2023-11-24T06:29:12.328224+0000","last_became_active":"2023-11-24T06:29:11.313762+0000","last_became_peered":"2023-11-24T06:29:11.313762+0000","last_unstale":"2023-11-24T06:29:12.328224+0000","last_undegraded":"2023-11-24T06:29:12.328224+0000","last_fullsized":"2023-11-24T06:29:12.328224+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:18.092054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328557+0000","last_change":"2023-11-24T06:29:11.290508+0000","last_active":"2023-11-24T06:29:12.328557+0000","last_peered":"2023-11-24T06:29:12.328557+0000","last_clean":"2023-11-24T06:29:12.328557+0000","last_became_active":"2023-11-24T06:29:11.290238+0000","last_became_peered":"2023-11-24T06:29:11.290238+0000","last_unstale":"2023-11-24T06:29:12.328557+0000","last_undegraded":"2023-11-24T06:29:12.328557+0000","last_fullsized":"2023-11-24T06:29:12.328557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:06.364499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683598+0000","last_change":"2023-11-24T06:29:11.301367+0000","last_active":"2023-11-24T06:29:12.683598+0000","last_peered":"2023-11-24T06:29:12.683598+0000","last_clean":"2023-11-24T06:29:12.683598+0000","last_became_active":"2023-11-24T06:29:11.300442+0000","last_became_peered":"2023-11-24T06:29:11.300442+0000","last_unstale":"2023-11-24T06:29:12.683598+0000","last_undegraded":"2023-11-24T06:29:12.683598+0000","last_fullsized":"2023-11-24T06:29:12.683598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:26:28.634476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683234+0000","last_change":"2023-11-24T06:29:11.301305+0000","last_active":"2023-11-24T06:29:12.683234+0000","last_peered":"2023-11-24T06:29:12.683234+0000","last_clean":"2023-11-24T06:29:12.683234+0000","last_became_active":"2023-11-24T06:29:11.300562+0000","last_became_peered":"2023-11-24T06:29:11.300562+0000","last_unstale":"2023-11-24T06:29:12.683234+0000","last_undegraded":"2023-11-24T06:29:12.683234+0000","last_fullsized":"2023-11-24T06:29:12.683234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:00.659984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285928+0000","last_change":"2023-11-24T06:29:11.288291+0000","last_active":"2023-11-24T06:29:12.285928+0000","last_peered":"2023-11-24T06:29:12.285928+0000","last_clean":"2023-11-24T06:29:12.285928+0000","last_became_active":"2023-11-24T06:29:11.287949+0000","last_became_peered":"2023-11-24T06:29:11.287949+0000","last_unstale":"2023-11-24T06:29:12.285928+0000","last_undegraded":"2023-11-24T06:29:12.285928+0000","last_fullsized":"2023-11-24T06:29:12.285928+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:10.213036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288479+0000","last_change":"2023-11-24T06:29:11.291383+0000","last_active":"2023-11-24T06:29:12.288479+0000","last_peered":"2023-11-24T06:29:12.288479+0000","last_clean":"2023-11-24T06:29:12.288479+0000","last_became_active":"2023-11-24T06:29:11.288825+0000","last_became_peered":"2023-11-24T06:29:11.288825+0000","last_unstale":"2023-11-24T06:29:12.288479+0000","last_undegraded":"2023-11-24T06:29:12.288479+0000","last_fullsized":"2023-11-24T06:29:12.288479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:13.161846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289203+0000","last_change":"2023-11-24T06:29:11.310994+0000","last_active":"2023-11-24T06:29:12.289203+0000","last_peered":"2023-11-24T06:29:12.289203+0000","last_clean":"2023-11-24T06:29:12.289203+0000","last_became_active":"2023-11-24T06:29:11.310647+0000","last_became_peered":"2023-11-24T06:29:11.310647+0000","last_unstale":"2023-11-24T06:29:12.289203+0000","last_undegraded":"2023-11-24T06:29:12.289203+0000","last_fullsized":"2023-11-24T06:29:12.289203+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:29.315454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681914+0000","last_change":"2023-11-24T06:29:11.285669+0000","last_active":"2023-11-24T06:29:12.681914+0000","last_peered":"2023-11-24T06:29:12.681914+0000","last_clean":"2023-11-24T06:29:12.681914+0000","last_became_active":"2023-11-24T06:29:11.285388+0000","last_became_peered":"2023-11-24T06:29:11.285388+0000","last_unstale":"2023-11-24T06:29:12.681914+0000","last_undegraded":"2023-11-24T06:29:12.681914+0000","last_fullsized":"2023-11-24T06:29:12.681914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:47.628375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288824+0000","last_change":"2023-11-24T06:29:11.286815+0000","last_active":"2023-11-24T06:29:12.288824+0000","last_peered":"2023-11-24T06:29:12.288824+0000","last_clean":"2023-11-24T06:29:12.288824+0000","last_became_active":"2023-11-24T06:29:11.286466+0000","last_became_peered":"2023-11-24T06:29:11.286466+0000","last_unstale":"2023-11-24T06:29:12.288824+0000","last_undegraded":"2023-11-24T06:29:12.288824+0000","last_fullsized":"2023-11-24T06:29:12.288824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:14:19.485966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681218+0000","last_change":"2023-11-24T06:29:11.280784+0000","last_active":"2023-11-24T06:29:12.681218+0000","last_peered":"2023-11-24T06:29:12.681218+0000","last_clean":"2023-11-24T06:29:12.681218+0000","last_became_active":"2023-11-24T06:29:11.280144+0000","last_became_peered":"2023-11-24T06:29:11.280144+0000","last_unstale":"2023-11-24T06:29:12.681218+0000","last_undegraded":"2023-11-24T06:29:12.681218+0000","last_fullsized":"2023-11-24T06:29:12.681218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:19.766174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683947+0000","last_change":"2023-11-24T06:29:11.300033+0000","last_active":"2023-11-24T06:29:12.683947+0000","last_peered":"2023-11-24T06:29:12.683947+0000","last_clean":"2023-11-24T06:29:12.683947+0000","last_became_active":"2023-11-24T06:29:11.299009+0000","last_became_peered":"2023-11-24T06:29:11.299009+0000","last_unstale":"2023-11-24T06:29:12.683947+0000","last_undegraded":"2023-11-24T06:29:12.683947+0000","last_fullsized":"2023-11-24T06:29:12.683947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:48.761991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681879+0000","last_change":"2023-11-24T06:29:11.292692+0000","last_active":"2023-11-24T06:29:12.681879+0000","last_peered":"2023-11-24T06:29:12.681879+0000","last_clean":"2023-11-24T06:29:12.681879+0000","last_became_active":"2023-11-24T06:29:11.292407+0000","last_became_peered":"2023-11-24T06:29:11.292407+0000","last_unstale":"2023-11-24T06:29:12.681879+0000","last_undegraded":"2023-11-24T06:29:12.681879+0000","last_fullsized":"2023-11-24T06:29:12.681879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:24.044724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290490+0000","last_change":"2023-11-24T06:29:11.289281+0000","last_active":"2023-11-24T06:29:12.290490+0000","last_peered":"2023-11-24T06:29:12.290490+0000","last_clean":"2023-11-24T06:29:12.290490+0000","last_became_active":"2023-11-24T06:29:11.288986+0000","last_became_peered":"2023-11-24T06:29:11.288986+0000","last_unstale":"2023-11-24T06:29:12.290490+0000","last_undegraded":"2023-11-24T06:29:12.290490+0000","last_fullsized":"2023-11-24T06:29:12.290490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:30.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288898+0000","last_change":"2023-11-24T06:29:11.310739+0000","last_active":"2023-11-24T06:29:12.288898+0000","last_peered":"2023-11-24T06:29:12.288898+0000","last_clean":"2023-11-24T06:29:12.288898+0000","last_became_active":"2023-11-24T06:29:11.310411+0000","last_became_peered":"2023-11-24T06:29:11.310411+0000","last_unstale":"2023-11-24T06:29:12.288898+0000","last_undegraded":"2023-11-24T06:29:12.288898+0000","last_fullsized":"2023-11-24T06:29:12.288898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:14:19.485966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327706+0000","last_change":"2023-11-24T06:29:11.284502+0000","last_active":"2023-11-24T06:29:12.327706+0000","last_peered":"2023-11-24T06:29:12.327706+0000","last_clean":"2023-11-24T06:29:12.327706+0000","last_became_active":"2023-11-24T06:29:11.283395+0000","last_became_peered":"2023-11-24T06:29:11.283395+0000","last_unstale":"2023-11-24T06:29:12.327706+0000","last_undegraded":"2023-11-24T06:29:12.327706+0000","last_fullsized":"2023-11-24T06:29:12.327706+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:38.902004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681367+0000","last_change":"2023-11-24T06:29:11.283330+0000","last_active":"2023-11-24T06:29:12.681367+0000","last_peered":"2023-11-24T06:29:12.681367+0000","last_clean":"2023-11-24T06:29:12.681367+0000","last_became_active":"2023-11-24T06:29:11.283039+0000","last_became_peered":"2023-11-24T06:29:11.283039+0000","last_unstale":"2023-11-24T06:29:12.681367+0000","last_undegraded":"2023-11-24T06:29:12.681367+0000","last_fullsized":"2023-11-24T06:29:12.681367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:38.862116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327402+0000","last_change":"2023-11-24T06:29:11.285666+0000","last_active":"2023-11-24T06:29:12.327402+0000","last_peered":"2023-11-24T06:29:12.327402+0000","last_clean":"2023-11-24T06:29:12.327402+0000","last_became_active":"2023-11-24T06:29:11.282590+0000","last_became_peered":"2023-11-24T06:29:11.282590+0000","last_unstale":"2023-11-24T06:29:12.327402+0000","last_undegraded":"2023-11-24T06:29:12.327402+0000","last_fullsized":"2023-11-24T06:29:12.327402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:14:19.485966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682665+0000","last_change":"2023-11-24T06:29:11.290739+0000","last_active":"2023-11-24T06:29:12.682665+0000","last_peered":"2023-11-24T06:29:12.682665+0000","last_clean":"2023-11-24T06:29:12.682665+0000","last_became_active":"2023-11-24T06:29:11.281388+0000","last_became_peered":"2023-11-24T06:29:11.281388+0000","last_unstale":"2023-11-24T06:29:12.682665+0000","last_undegraded":"2023-11-24T06:29:12.682665+0000","last_fullsized":"2023-11-24T06:29:12.682665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:49:20.918194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681915+0000","last_change":"2023-11-24T06:29:10.281226+0000","last_active":"2023-11-24T06:29:12.681915+0000","last_peered":"2023-11-24T06:29:12.681915+0000","last_clean":"2023-11-24T06:29:12.681915+0000","last_became_active":"2023-11-24T06:29:10.280942+0000","last_became_peered":"2023-11-24T06:29:10.280942+0000","last_unstale":"2023-11-24T06:29:12.681915+0000","last_undegraded":"2023-11-24T06:29:12.681915+0000","last_fullsized":"2023-11-24T06:29:12.681915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:07:22.512103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681886+0000","last_change":"2023-11-24T06:29:11.285786+0000","last_active":"2023-11-24T06:29:12.681886+0000","last_peered":"2023-11-24T06:29:12.681886+0000","last_clean":"2023-11-24T06:29:12.681886+0000","last_became_active":"2023-11-24T06:29:11.284070+0000","last_became_peered":"2023-11-24T06:29:11.284070+0000","last_unstale":"2023-11-24T06:29:12.681886+0000","last_undegraded":"2023-11-24T06:29:12.681886+0000","last_fullsized":"2023-11-24T06:29:12.681886+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:52.500523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682794+0000","last_change":"2023-11-24T06:29:10.282192+0000","last_active":"2023-11-24T06:29:12.682794+0000","last_peered":"2023-11-24T06:29:12.682794+0000","last_clean":"2023-11-24T06:29:12.682794+0000","last_became_active":"2023-11-24T06:29:10.281906+0000","last_became_peered":"2023-11-24T06:29:10.281906+0000","last_unstale":"2023-11-24T06:29:12.682794+0000","last_undegraded":"2023-11-24T06:29:12.682794+0000","last_fullsized":"2023-11-24T06:29:12.682794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:30.133018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289478+0000","last_change":"2023-11-24T06:29:11.310741+0000","last_active":"2023-11-24T06:29:12.289478+0000","last_peered":"2023-11-24T06:29:12.289478+0000","last_clean":"2023-11-24T06:29:12.289478+0000","last_became_active":"2023-11-24T06:29:11.310445+0000","last_became_peered":"2023-11-24T06:29:11.310445+0000","last_unstale":"2023-11-24T06:29:12.289478+0000","last_undegraded":"2023-11-24T06:29:12.289478+0000","last_fullsized":"2023-11-24T06:29:12.289478+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:38.902004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683413+0000","last_change":"2023-11-24T06:29:10.283612+0000","last_active":"2023-11-24T06:29:12.683413+0000","last_peered":"2023-11-24T06:29:12.683413+0000","last_clean":"2023-11-24T06:29:12.683413+0000","last_became_active":"2023-11-24T06:29:10.283281+0000","last_became_peered":"2023-11-24T06:29:10.283281+0000","last_unstale":"2023-11-24T06:29:12.683413+0000","last_undegraded":"2023-11-24T06:29:12.683413+0000","last_fullsized":"2023-11-24T06:29:12.683413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:53.896209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682225+0000","last_change":"2023-11-24T06:29:11.285921+0000","last_active":"2023-11-24T06:29:12.682225+0000","last_peered":"2023-11-24T06:29:12.682225+0000","last_clean":"2023-11-24T06:29:12.682225+0000","last_became_active":"2023-11-24T06:29:11.285457+0000","last_became_peered":"2023-11-24T06:29:11.285457+0000","last_unstale":"2023-11-24T06:29:12.682225+0000","last_undegraded":"2023-11-24T06:29:12.682225+0000","last_fullsized":"2023-11-24T06:29:12.682225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:25.204869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286065+0000","last_change":"2023-11-24T06:29:10.273947+0000","last_active":"2023-11-24T06:29:12.286065+0000","last_peered":"2023-11-24T06:29:12.286065+0000","last_clean":"2023-11-24T06:29:12.286065+0000","last_became_active":"2023-11-24T06:29:10.273640+0000","last_became_peered":"2023-11-24T06:29:10.273640+0000","last_unstale":"2023-11-24T06:29:12.286065+0000","last_undegraded":"2023-11-24T06:29:12.286065+0000","last_fullsized":"2023-11-24T06:29:12.286065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:43.191205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328361+0000","last_change":"2023-11-24T06:29:11.281017+0000","last_active":"2023-11-24T06:29:12.328361+0000","last_peered":"2023-11-24T06:29:12.328361+0000","last_clean":"2023-11-24T06:29:12.328361+0000","last_became_active":"2023-11-24T06:29:11.280434+0000","last_became_peered":"2023-11-24T06:29:11.280434+0000","last_unstale":"2023-11-24T06:29:12.328361+0000","last_undegraded":"2023-11-24T06:29:12.328361+0000","last_fullsized":"2023-11-24T06:29:12.328361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:10.384116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290254+0000","last_change":"2023-11-24T06:29:10.278551+0000","last_active":"2023-11-24T06:29:12.290254+0000","last_peered":"2023-11-24T06:29:12.290254+0000","last_clean":"2023-11-24T06:29:12.290254+0000","last_became_active":"2023-11-24T06:29:10.278253+0000","last_became_peered":"2023-11-24T06:29:10.278253+0000","last_unstale":"2023-11-24T06:29:12.290254+0000","last_undegraded":"2023-11-24T06:29:12.290254+0000","last_fullsized":"2023-11-24T06:29:12.290254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:56.397081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290282+0000","last_change":"2023-11-24T06:29:11.295838+0000","last_active":"2023-11-24T06:29:12.290282+0000","last_peered":"2023-11-24T06:29:12.290282+0000","last_clean":"2023-11-24T06:29:12.290282+0000","last_became_active":"2023-11-24T06:29:11.295247+0000","last_became_peered":"2023-11-24T06:29:11.295247+0000","last_unstale":"2023-11-24T06:29:12.290282+0000","last_undegraded":"2023-11-24T06:29:12.290282+0000","last_fullsized":"2023-11-24T06:29:12.290282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:14.532797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289443+0000","last_change":"2023-11-24T06:29:10.284802+0000","last_active":"2023-11-24T06:29:12.289443+0000","last_peered":"2023-11-24T06:29:12.289443+0000","last_clean":"2023-11-24T06:29:12.289443+0000","last_became_active":"2023-11-24T06:29:10.283754+0000","last_became_peered":"2023-11-24T06:29:10.283754+0000","last_unstale":"2023-11-24T06:29:12.289443+0000","last_undegraded":"2023-11-24T06:29:12.289443+0000","last_fullsized":"2023-11-24T06:29:12.289443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:09:59.719383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288765+0000","last_change":"2023-11-24T06:29:11.285574+0000","last_active":"2023-11-24T06:29:12.288765+0000","last_peered":"2023-11-24T06:29:12.288765+0000","last_clean":"2023-11-24T06:29:12.288765+0000","last_became_active":"2023-11-24T06:29:11.285257+0000","last_became_peered":"2023-11-24T06:29:11.285257+0000","last_unstale":"2023-11-24T06:29:12.288765+0000","last_undegraded":"2023-11-24T06:29:12.288765+0000","last_fullsized":"2023-11-24T06:29:12.288765+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:36.625390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680887+0000","last_change":"2023-11-24T06:29:10.265611+0000","last_active":"2023-11-24T06:29:12.680887+0000","last_peered":"2023-11-24T06:29:12.680887+0000","last_clean":"2023-11-24T06:29:12.680887+0000","last_became_active":"2023-11-24T06:29:10.265275+0000","last_became_peered":"2023-11-24T06:29:10.265275+0000","last_unstale":"2023-11-24T06:29:12.680887+0000","last_undegraded":"2023-11-24T06:29:12.680887+0000","last_fullsized":"2023-11-24T06:29:12.680887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:04.116234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680637+0000","last_change":"2023-11-24T06:29:11.282332+0000","last_active":"2023-11-24T06:29:12.680637+0000","last_peered":"2023-11-24T06:29:12.680637+0000","last_clean":"2023-11-24T06:29:12.680637+0000","last_became_active":"2023-11-24T06:29:11.282064+0000","last_became_peered":"2023-11-24T06:29:11.282064+0000","last_unstale":"2023-11-24T06:29:12.680637+0000","last_undegraded":"2023-11-24T06:29:12.680637+0000","last_fullsized":"2023-11-24T06:29:12.680637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:57.678666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288307+0000","last_change":"2023-11-24T06:29:10.276033+0000","last_active":"2023-11-24T06:29:12.288307+0000","last_peered":"2023-11-24T06:29:12.288307+0000","last_clean":"2023-11-24T06:29:12.288307+0000","last_became_active":"2023-11-24T06:29:10.275731+0000","last_became_peered":"2023-11-24T06:29:10.275731+0000","last_unstale":"2023-11-24T06:29:12.288307+0000","last_undegraded":"2023-11-24T06:29:12.288307+0000","last_fullsized":"2023-11-24T06:29:12.288307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:05.645745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.291198+0000","last_change":"2023-11-24T06:29:11.310394+0000","last_active":"2023-11-24T06:29:12.291198+0000","last_peered":"2023-11-24T06:29:12.291198+0000","last_clean":"2023-11-24T06:29:12.291198+0000","last_became_active":"2023-11-24T06:29:11.309916+0000","last_became_peered":"2023-11-24T06:29:11.309916+0000","last_unstale":"2023-11-24T06:29:12.291198+0000","last_undegraded":"2023-11-24T06:29:12.291198+0000","last_fullsized":"2023-11-24T06:29:12.291198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:10.384116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683678+0000","last_change":"2023-11-24T06:29:10.284021+0000","last_active":"2023-11-24T06:29:12.683678+0000","last_peered":"2023-11-24T06:29:12.683678+0000","last_clean":"2023-11-24T06:29:12.683678+0000","last_became_active":"2023-11-24T06:29:10.283345+0000","last_became_peered":"2023-11-24T06:29:10.283345+0000","last_unstale":"2023-11-24T06:29:12.683678+0000","last_undegraded":"2023-11-24T06:29:12.683678+0000","last_fullsized":"2023-11-24T06:29:12.683678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:42.009905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683567+0000","last_change":"2023-11-24T06:29:11.300344+0000","last_active":"2023-11-24T06:29:12.683567+0000","last_peered":"2023-11-24T06:29:12.683567+0000","last_clean":"2023-11-24T06:29:12.683567+0000","last_became_active":"2023-11-24T06:29:11.299842+0000","last_became_peered":"2023-11-24T06:29:11.299842+0000","last_unstale":"2023-11-24T06:29:12.683567+0000","last_undegraded":"2023-11-24T06:29:12.683567+0000","last_fullsized":"2023-11-24T06:29:12.683567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:14.542488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682715+0000","last_change":"2023-11-24T06:29:10.269271+0000","last_active":"2023-11-24T06:29:12.682715+0000","last_peered":"2023-11-24T06:29:12.682715+0000","last_clean":"2023-11-24T06:29:12.682715+0000","last_became_active":"2023-11-24T06:29:10.269002+0000","last_became_peered":"2023-11-24T06:29:10.269002+0000","last_unstale":"2023-11-24T06:29:12.682715+0000","last_undegraded":"2023-11-24T06:29:12.682715+0000","last_fullsized":"2023-11-24T06:29:12.682715+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:43.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683085+0000","last_change":"2023-11-24T06:29:11.308920+0000","last_active":"2023-11-24T06:29:12.683085+0000","last_peered":"2023-11-24T06:29:12.683085+0000","last_clean":"2023-11-24T06:29:12.683085+0000","last_became_active":"2023-11-24T06:29:11.305891+0000","last_became_peered":"2023-11-24T06:29:11.305891+0000","last_unstale":"2023-11-24T06:29:12.683085+0000","last_undegraded":"2023-11-24T06:29:12.683085+0000","last_fullsized":"2023-11-24T06:29:12.683085+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:46.529579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290812+0000","last_change":"2023-11-24T06:29:10.284433+0000","last_active":"2023-11-24T06:29:12.290812+0000","last_peered":"2023-11-24T06:29:12.290812+0000","last_clean":"2023-11-24T06:29:12.290812+0000","last_became_active":"2023-11-24T06:29:10.282509+0000","last_became_peered":"2023-11-24T06:29:10.282509+0000","last_unstale":"2023-11-24T06:29:12.290812+0000","last_undegraded":"2023-11-24T06:29:12.290812+0000","last_fullsized":"2023-11-24T06:29:12.290812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:47.226931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290475+0000","last_change":"2023-11-24T06:29:11.298859+0000","last_active":"2023-11-24T06:29:12.290475+0000","last_peered":"2023-11-24T06:29:12.290475+0000","last_clean":"2023-11-24T06:29:12.290475+0000","last_became_active":"2023-11-24T06:29:11.298366+0000","last_became_peered":"2023-11-24T06:29:11.298366+0000","last_unstale":"2023-11-24T06:29:12.290475+0000","last_undegraded":"2023-11-24T06:29:12.290475+0000","last_fullsized":"2023-11-24T06:29:12.290475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:38.902004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683162+0000","last_change":"2023-11-24T06:29:10.283604+0000","last_active":"2023-11-24T06:29:12.683162+0000","last_peered":"2023-11-24T06:29:12.683162+0000","last_clean":"2023-11-24T06:29:12.683162+0000","last_became_active":"2023-11-24T06:29:10.283259+0000","last_became_peered":"2023-11-24T06:29:10.283259+0000","last_unstale":"2023-11-24T06:29:12.683162+0000","last_undegraded":"2023-11-24T06:29:12.683162+0000","last_fullsized":"2023-11-24T06:29:12.683162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:14.103320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680475+0000","last_change":"2023-11-24T06:29:11.284438+0000","last_active":"2023-11-24T06:29:12.680475+0000","last_peered":"2023-11-24T06:29:12.680475+0000","last_clean":"2023-11-24T06:29:12.680475+0000","last_became_active":"2023-11-24T06:29:11.283701+0000","last_became_peered":"2023-11-24T06:29:11.283701+0000","last_unstale":"2023-11-24T06:29:12.680475+0000","last_undegraded":"2023-11-24T06:29:12.680475+0000","last_fullsized":"2023-11-24T06:29:12.680475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:43.024477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289236+0000","last_change":"2023-11-24T06:29:10.277155+0000","last_active":"2023-11-24T06:29:12.289236+0000","last_peered":"2023-11-24T06:29:12.289236+0000","last_clean":"2023-11-24T06:29:12.289236+0000","last_became_active":"2023-11-24T06:29:10.274961+0000","last_became_peered":"2023-11-24T06:29:10.274961+0000","last_unstale":"2023-11-24T06:29:12.289236+0000","last_undegraded":"2023-11-24T06:29:12.289236+0000","last_fullsized":"2023-11-24T06:29:12.289236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:30.133018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682175+0000","last_change":"2023-11-24T06:29:11.308955+0000","last_active":"2023-11-24T06:29:12.682175+0000","last_peered":"2023-11-24T06:29:12.682175+0000","last_clean":"2023-11-24T06:29:12.682175+0000","last_became_active":"2023-11-24T06:29:11.308662+0000","last_became_peered":"2023-11-24T06:29:11.308662+0000","last_unstale":"2023-11-24T06:29:12.682175+0000","last_undegraded":"2023-11-24T06:29:12.682175+0000","last_fullsized":"2023-11-24T06:29:12.682175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:12.393016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680573+0000","last_change":"2023-11-24T06:29:10.278376+0000","last_active":"2023-11-24T06:29:12.680573+0000","last_peered":"2023-11-24T06:29:12.680573+0000","last_clean":"2023-11-24T06:29:12.680573+0000","last_became_active":"2023-11-24T06:29:10.278051+0000","last_became_peered":"2023-11-24T06:29:10.278051+0000","last_unstale":"2023-11-24T06:29:12.680573+0000","last_undegraded":"2023-11-24T06:29:12.680573+0000","last_fullsized":"2023-11-24T06:29:12.680573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:36.148824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327704+0000","last_change":"2023-11-24T06:29:11.284085+0000","last_active":"2023-11-24T06:29:12.327704+0000","last_peered":"2023-11-24T06:29:12.327704+0000","last_clean":"2023-11-24T06:29:12.327704+0000","last_became_active":"2023-11-24T06:29:11.283771+0000","last_became_peered":"2023-11-24T06:29:11.283771+0000","last_unstale":"2023-11-24T06:29:12.327704+0000","last_undegraded":"2023-11-24T06:29:12.327704+0000","last_fullsized":"2023-11-24T06:29:12.327704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:14.532797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680893+0000","last_change":"2023-11-24T06:29:10.279929+0000","last_active":"2023-11-24T06:29:12.680893+0000","last_peered":"2023-11-24T06:29:12.680893+0000","last_clean":"2023-11-24T06:29:12.680893+0000","last_became_active":"2023-11-24T06:29:10.279575+0000","last_became_peered":"2023-11-24T06:29:10.279575+0000","last_unstale":"2023-11-24T06:29:12.680893+0000","last_undegraded":"2023-11-24T06:29:12.680893+0000","last_fullsized":"2023-11-24T06:29:12.680893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:42.651516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681482+0000","last_change":"2023-11-24T06:29:11.284874+0000","last_active":"2023-11-24T06:29:12.681482+0000","last_peered":"2023-11-24T06:29:12.681482+0000","last_clean":"2023-11-24T06:29:12.681482+0000","last_became_active":"2023-11-24T06:29:11.284391+0000","last_became_peered":"2023-11-24T06:29:11.284391+0000","last_unstale":"2023-11-24T06:29:12.681482+0000","last_undegraded":"2023-11-24T06:29:12.681482+0000","last_fullsized":"2023-11-24T06:29:12.681482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.216950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680419+0000","last_change":"2023-11-24T06:29:10.284791+0000","last_active":"2023-11-24T06:29:12.680419+0000","last_peered":"2023-11-24T06:29:12.680419+0000","last_clean":"2023-11-24T06:29:12.680419+0000","last_became_active":"2023-11-24T06:29:10.284485+0000","last_became_peered":"2023-11-24T06:29:10.284485+0000","last_unstale":"2023-11-24T06:29:12.680419+0000","last_undegraded":"2023-11-24T06:29:12.680419+0000","last_fullsized":"2023-11-24T06:29:12.680419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.751601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289326+0000","last_change":"2023-11-24T06:29:11.282035+0000","last_active":"2023-11-24T06:29:12.289326+0000","last_peered":"2023-11-24T06:29:12.289326+0000","last_clean":"2023-11-24T06:29:12.289326+0000","last_became_active":"2023-11-24T06:29:11.281762+0000","last_became_peered":"2023-11-24T06:29:11.281762+0000","last_unstale":"2023-11-24T06:29:12.289326+0000","last_undegraded":"2023-11-24T06:29:12.289326+0000","last_fullsized":"2023-11-24T06:29:12.289326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:14.532797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681363+0000","last_change":"2023-11-24T06:29:10.276069+0000","last_active":"2023-11-24T06:29:12.681363+0000","last_peered":"2023-11-24T06:29:12.681363+0000","last_clean":"2023-11-24T06:29:12.681363+0000","last_became_active":"2023-11-24T06:29:10.275767+0000","last_became_peered":"2023-11-24T06:29:10.275767+0000","last_unstale":"2023-11-24T06:29:12.681363+0000","last_undegraded":"2023-11-24T06:29:12.681363+0000","last_fullsized":"2023-11-24T06:29:12.681363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:07.278001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.291184+0000","last_change":"2023-11-24T06:29:11.285077+0000","last_active":"2023-11-24T06:29:12.291184+0000","last_peered":"2023-11-24T06:29:12.291184+0000","last_clean":"2023-11-24T06:29:12.291184+0000","last_became_active":"2023-11-24T06:29:11.284814+0000","last_became_peered":"2023-11-24T06:29:11.284814+0000","last_unstale":"2023-11-24T06:29:12.291184+0000","last_undegraded":"2023-11-24T06:29:12.291184+0000","last_fullsized":"2023-11-24T06:29:12.291184+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:48.087034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289601+0000","last_change":"2023-11-24T06:29:10.277251+0000","last_active":"2023-11-24T06:29:12.289601+0000","last_peered":"2023-11-24T06:29:12.289601+0000","last_clean":"2023-11-24T06:29:12.289601+0000","last_became_active":"2023-11-24T06:29:10.275293+0000","last_became_peered":"2023-11-24T06:29:10.275293+0000","last_unstale":"2023-11-24T06:29:12.289601+0000","last_undegraded":"2023-11-24T06:29:12.289601+0000","last_fullsized":"2023-11-24T06:29:12.289601+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:46.613802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681358+0000","last_change":"2023-11-24T06:29:11.292649+0000","last_active":"2023-11-24T06:29:12.681358+0000","last_peered":"2023-11-24T06:29:12.681358+0000","last_clean":"2023-11-24T06:29:12.681358+0000","last_became_active":"2023-11-24T06:29:11.292358+0000","last_became_peered":"2023-11-24T06:29:11.292358+0000","last_unstale":"2023-11-24T06:29:12.681358+0000","last_undegraded":"2023-11-24T06:29:12.681358+0000","last_fullsized":"2023-11-24T06:29:12.681358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:34.786099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289138+0000","last_change":"2023-11-24T06:29:10.271755+0000","last_active":"2023-11-24T06:29:12.289138+0000","last_peered":"2023-11-24T06:29:12.289138+0000","last_clean":"2023-11-24T06:29:12.289138+0000","last_became_active":"2023-11-24T06:29:10.271435+0000","last_became_peered":"2023-11-24T06:29:10.271435+0000","last_unstale":"2023-11-24T06:29:12.289138+0000","last_undegraded":"2023-11-24T06:29:12.289138+0000","last_fullsized":"2023-11-24T06:29:12.289138+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:51.485950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286034+0000","last_change":"2023-11-24T06:29:11.282118+0000","last_active":"2023-11-24T06:29:12.286034+0000","last_peered":"2023-11-24T06:29:12.286034+0000","last_clean":"2023-11-24T06:29:12.286034+0000","last_became_active":"2023-11-24T06:29:11.281574+0000","last_became_peered":"2023-11-24T06:29:11.281574+0000","last_unstale":"2023-11-24T06:29:12.286034+0000","last_undegraded":"2023-11-24T06:29:12.286034+0000","last_fullsized":"2023-11-24T06:29:12.286034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:00.733955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289686+0000","last_change":"2023-11-24T06:29:10.284750+0000","last_active":"2023-11-24T06:29:12.289686+0000","last_peered":"2023-11-24T06:29:12.289686+0000","last_clean":"2023-11-24T06:29:12.289686+0000","last_became_active":"2023-11-24T06:29:10.284466+0000","last_became_peered":"2023-11-24T06:29:10.284466+0000","last_unstale":"2023-11-24T06:29:12.289686+0000","last_undegraded":"2023-11-24T06:29:12.289686+0000","last_fullsized":"2023-11-24T06:29:12.289686+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:37.847544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285546+0000","last_change":"2023-11-24T06:29:11.281785+0000","last_active":"2023-11-24T06:29:12.285546+0000","last_peered":"2023-11-24T06:29:12.285546+0000","last_clean":"2023-11-24T06:29:12.285546+0000","last_became_active":"2023-11-24T06:29:11.279634+0000","last_became_peered":"2023-11-24T06:29:11.279634+0000","last_unstale":"2023-11-24T06:29:12.285546+0000","last_undegraded":"2023-11-24T06:29:12.285546+0000","last_fullsized":"2023-11-24T06:29:12.285546+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:31.147590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682752+0000","last_change":"2023-11-24T06:29:10.274576+0000","last_active":"2023-11-24T06:29:12.682752+0000","last_peered":"2023-11-24T06:29:12.682752+0000","last_clean":"2023-11-24T06:29:12.682752+0000","last_became_active":"2023-11-24T06:29:10.274244+0000","last_became_peered":"2023-11-24T06:29:10.274244+0000","last_unstale":"2023-11-24T06:29:12.682752+0000","last_undegraded":"2023-11-24T06:29:12.682752+0000","last_fullsized":"2023-11-24T06:29:12.682752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:45.022343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286210+0000","last_change":"2023-11-24T06:29:11.295929+0000","last_active":"2023-11-24T06:29:12.286210+0000","last_peered":"2023-11-24T06:29:12.286210+0000","last_clean":"2023-11-24T06:29:12.286210+0000","last_became_active":"2023-11-24T06:29:11.295404+0000","last_became_peered":"2023-11-24T06:29:11.295404+0000","last_unstale":"2023-11-24T06:29:12.286210+0000","last_undegraded":"2023-11-24T06:29:12.286210+0000","last_fullsized":"2023-11-24T06:29:12.286210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:46.036916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681399+0000","last_change":"2023-11-24T06:29:10.279578+0000","last_active":"2023-11-24T06:29:12.681399+0000","last_peered":"2023-11-24T06:29:12.681399+0000","last_clean":"2023-11-24T06:29:12.681399+0000","last_became_active":"2023-11-24T06:29:10.279224+0000","last_became_peered":"2023-11-24T06:29:10.279224+0000","last_unstale":"2023-11-24T06:29:12.681399+0000","last_undegraded":"2023-11-24T06:29:12.681399+0000","last_fullsized":"2023-11-24T06:29:12.681399+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:46.064061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285855+0000","last_change":"2023-11-24T06:29:11.291015+0000","last_active":"2023-11-24T06:29:12.285855+0000","last_peered":"2023-11-24T06:29:12.285855+0000","last_clean":"2023-11-24T06:29:12.285855+0000","last_became_active":"2023-11-24T06:29:11.290743+0000","last_became_peered":"2023-11-24T06:29:11.290743+0000","last_unstale":"2023-11-24T06:29:12.285855+0000","last_undegraded":"2023-11-24T06:29:12.285855+0000","last_fullsized":"2023-11-24T06:29:12.285855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:47.078633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288264+0000","last_change":"2023-11-24T06:29:10.280791+0000","last_active":"2023-11-24T06:29:12.288264+0000","last_peered":"2023-11-24T06:29:12.288264+0000","last_clean":"2023-11-24T06:29:12.288264+0000","last_became_active":"2023-11-24T06:29:10.280032+0000","last_became_peered":"2023-11-24T06:29:10.280032+0000","last_unstale":"2023-11-24T06:29:12.288264+0000","last_undegraded":"2023-11-24T06:29:12.288264+0000","last_fullsized":"2023-11-24T06:29:12.288264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:45.022343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327522+0000","last_change":"2023-11-24T06:29:11.287226+0000","last_active":"2023-11-24T06:29:12.327522+0000","last_peered":"2023-11-24T06:29:12.327522+0000","last_clean":"2023-11-24T06:29:12.327522+0000","last_became_active":"2023-11-24T06:29:11.284025+0000","last_became_peered":"2023-11-24T06:29:11.284025+0000","last_unstale":"2023-11-24T06:29:12.327522+0000","last_undegraded":"2023-11-24T06:29:12.327522+0000","last_fullsized":"2023-11-24T06:29:12.327522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:29.315454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288390+0000","last_change":"2023-11-24T06:29:10.283882+0000","last_active":"2023-11-24T06:29:12.288390+0000","last_peered":"2023-11-24T06:29:12.288390+0000","last_clean":"2023-11-24T06:29:12.288390+0000","last_became_active":"2023-11-24T06:29:10.283591+0000","last_became_peered":"2023-11-24T06:29:10.283591+0000","last_unstale":"2023-11-24T06:29:12.288390+0000","last_undegraded":"2023-11-24T06:29:12.288390+0000","last_fullsized":"2023-11-24T06:29:12.288390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:45.022343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285761+0000","last_change":"2023-11-24T06:29:11.277568+0000","last_active":"2023-11-24T06:29:12.285761+0000","last_peered":"2023-11-24T06:29:12.285761+0000","last_clean":"2023-11-24T06:29:12.285761+0000","last_became_active":"2023-11-24T06:29:11.277383+0000","last_became_peered":"2023-11-24T06:29:11.277383+0000","last_unstale":"2023-11-24T06:29:12.285761+0000","last_undegraded":"2023-11-24T06:29:12.285761+0000","last_fullsized":"2023-11-24T06:29:12.285761+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:44.174589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681416+0000","last_change":"2023-11-24T06:29:10.279864+0000","last_active":"2023-11-24T06:29:12.681416+0000","last_peered":"2023-11-24T06:29:12.681416+0000","last_clean":"2023-11-24T06:29:12.681416+0000","last_became_active":"2023-11-24T06:29:10.279484+0000","last_became_peered":"2023-11-24T06:29:10.279484+0000","last_unstale":"2023-11-24T06:29:12.681416+0000","last_undegraded":"2023-11-24T06:29:12.681416+0000","last_fullsized":"2023-11-24T06:29:12.681416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:33:05.543857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.291358+0000","last_change":"2023-11-24T06:29:11.293523+0000","last_active":"2023-11-24T06:29:12.291358+0000","last_peered":"2023-11-24T06:29:12.291358+0000","last_clean":"2023-11-24T06:29:12.291358+0000","last_became_active":"2023-11-24T06:29:11.293244+0000","last_became_peered":"2023-11-24T06:29:11.293244+0000","last_unstale":"2023-11-24T06:29:12.291358+0000","last_undegraded":"2023-11-24T06:29:12.291358+0000","last_fullsized":"2023-11-24T06:29:12.291358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:06.364499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682766+0000","last_change":"2023-11-24T06:29:10.277004+0000","last_active":"2023-11-24T06:29:12.682766+0000","last_peered":"2023-11-24T06:29:12.682766+0000","last_clean":"2023-11-24T06:29:12.682766+0000","last_became_active":"2023-11-24T06:29:10.276327+0000","last_became_peered":"2023-11-24T06:29:10.276327+0000","last_unstale":"2023-11-24T06:29:12.682766+0000","last_undegraded":"2023-11-24T06:29:12.682766+0000","last_fullsized":"2023-11-24T06:29:12.682766+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:46.064061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682524+0000","last_change":"2023-11-24T06:29:11.301942+0000","last_active":"2023-11-24T06:29:12.682524+0000","last_peered":"2023-11-24T06:29:12.682524+0000","last_clean":"2023-11-24T06:29:12.682524+0000","last_became_active":"2023-11-24T06:29:11.301716+0000","last_became_peered":"2023-11-24T06:29:11.301716+0000","last_unstale":"2023-11-24T06:29:12.682524+0000","last_undegraded":"2023-11-24T06:29:12.682524+0000","last_fullsized":"2023-11-24T06:29:12.682524+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:52.500523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681398+0000","last_change":"2023-11-24T06:29:10.267944+0000","last_active":"2023-11-24T06:29:12.681398+0000","last_peered":"2023-11-24T06:29:12.681398+0000","last_clean":"2023-11-24T06:29:12.681398+0000","last_became_active":"2023-11-24T06:29:10.267632+0000","last_became_peered":"2023-11-24T06:29:10.267632+0000","last_unstale":"2023-11-24T06:29:12.681398+0000","last_undegraded":"2023-11-24T06:29:12.681398+0000","last_fullsized":"2023-11-24T06:29:12.681398+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:42.651516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681315+0000","last_change":"2023-11-24T06:29:11.290691+0000","last_active":"2023-11-24T06:29:12.681315+0000","last_peered":"2023-11-24T06:29:12.681315+0000","last_clean":"2023-11-24T06:29:12.681315+0000","last_became_active":"2023-11-24T06:29:11.290400+0000","last_became_peered":"2023-11-24T06:29:11.290400+0000","last_unstale":"2023-11-24T06:29:12.681315+0000","last_undegraded":"2023-11-24T06:29:12.681315+0000","last_fullsized":"2023-11-24T06:29:12.681315+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:46.036916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682990+0000","last_change":"2023-11-24T06:29:10.272285+0000","last_active":"2023-11-24T06:29:12.682990+0000","last_peered":"2023-11-24T06:29:12.682990+0000","last_clean":"2023-11-24T06:29:12.682990+0000","last_became_active":"2023-11-24T06:29:10.271851+0000","last_became_peered":"2023-11-24T06:29:10.271851+0000","last_unstale":"2023-11-24T06:29:12.682990+0000","last_undegraded":"2023-11-24T06:29:12.682990+0000","last_fullsized":"2023-11-24T06:29:12.682990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:37.847544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682681+0000","last_change":"2023-11-24T06:29:11.293391+0000","last_active":"2023-11-24T06:29:12.682681+0000","last_peered":"2023-11-24T06:29:12.682681+0000","last_clean":"2023-11-24T06:29:12.682681+0000","last_became_active":"2023-11-24T06:29:11.292332+0000","last_became_peered":"2023-11-24T06:29:11.292332+0000","last_unstale":"2023-11-24T06:29:12.682681+0000","last_undegraded":"2023-11-24T06:29:12.682681+0000","last_fullsized":"2023-11-24T06:29:12.682681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:48.241504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680674+0000","last_change":"2023-11-24T06:29:10.279399+0000","last_active":"2023-11-24T06:29:12.680674+0000","last_peered":"2023-11-24T06:29:12.680674+0000","last_clean":"2023-11-24T06:29:12.680674+0000","last_became_active":"2023-11-24T06:29:10.278981+0000","last_became_peered":"2023-11-24T06:29:10.278981+0000","last_unstale":"2023-11-24T06:29:12.680674+0000","last_undegraded":"2023-11-24T06:29:12.680674+0000","last_fullsized":"2023-11-24T06:29:12.680674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:04.116234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680305+0000","last_change":"2023-11-24T06:29:11.310443+0000","last_active":"2023-11-24T06:29:12.680305+0000","last_peered":"2023-11-24T06:29:12.680305+0000","last_clean":"2023-11-24T06:29:12.680305+0000","last_became_active":"2023-11-24T06:29:11.310183+0000","last_became_peered":"2023-11-24T06:29:11.310183+0000","last_unstale":"2023-11-24T06:29:12.680305+0000","last_undegraded":"2023-11-24T06:29:12.680305+0000","last_fullsized":"2023-11-24T06:29:12.680305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:02.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288508+0000","last_change":"2023-11-24T06:29:10.270585+0000","last_active":"2023-11-24T06:29:12.288508+0000","last_peered":"2023-11-24T06:29:12.288508+0000","last_clean":"2023-11-24T06:29:12.288508+0000","last_became_active":"2023-11-24T06:29:10.270168+0000","last_became_peered":"2023-11-24T06:29:10.270168+0000","last_unstale":"2023-11-24T06:29:12.288508+0000","last_undegraded":"2023-11-24T06:29:12.288508+0000","last_fullsized":"2023-11-24T06:29:12.288508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:07:22.512103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288327+0000","last_change":"2023-11-24T06:29:11.306265+0000","last_active":"2023-11-24T06:29:12.288327+0000","last_peered":"2023-11-24T06:29:12.288327+0000","last_clean":"2023-11-24T06:29:12.288327+0000","last_became_active":"2023-11-24T06:29:11.305921+0000","last_became_peered":"2023-11-24T06:29:11.305921+0000","last_unstale":"2023-11-24T06:29:12.288327+0000","last_undegraded":"2023-11-24T06:29:12.288327+0000","last_fullsized":"2023-11-24T06:29:12.288327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:21.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680549+0000","last_change":"2023-11-24T06:29:10.282578+0000","last_active":"2023-11-24T06:29:12.680549+0000","last_peered":"2023-11-24T06:29:12.680549+0000","last_clean":"2023-11-24T06:29:12.680549+0000","last_became_active":"2023-11-24T06:29:10.282324+0000","last_became_peered":"2023-11-24T06:29:10.282324+0000","last_unstale":"2023-11-24T06:29:12.680549+0000","last_undegraded":"2023-11-24T06:29:12.680549+0000","last_fullsized":"2023-11-24T06:29:12.680549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:33.771527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327982+0000","last_change":"2023-11-24T06:29:11.280873+0000","last_active":"2023-11-24T06:29:12.327982+0000","last_peered":"2023-11-24T06:29:12.327982+0000","last_clean":"2023-11-24T06:29:12.327982+0000","last_became_active":"2023-11-24T06:29:11.279668+0000","last_became_peered":"2023-11-24T06:29:11.279668+0000","last_unstale":"2023-11-24T06:29:12.327982+0000","last_undegraded":"2023-11-24T06:29:12.327982+0000","last_fullsized":"2023-11-24T06:29:12.327982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:24.991791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681323+0000","last_change":"2023-11-24T06:29:10.276596+0000","last_active":"2023-11-24T06:29:12.681323+0000","last_peered":"2023-11-24T06:29:12.681323+0000","last_clean":"2023-11-24T06:29:12.681323+0000","last_became_active":"2023-11-24T06:29:10.276278+0000","last_became_peered":"2023-11-24T06:29:10.276278+0000","last_unstale":"2023-11-24T06:29:12.681323+0000","last_undegraded":"2023-11-24T06:29:12.681323+0000","last_fullsized":"2023-11-24T06:29:12.681323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:09.198463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:29:11.261762+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:29:11.261762+0000","last_peered":"2023-11-24T06:29:11.261762+0000","last_clean":"2023-11-24T06:29:11.261762+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:29:11.261762+0000","last_undegraded":"2023-11-24T06:29:11.261762+0000","last_fullsized":"2023-11-24T06:29:11.261762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":750119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.274}]},{"osd":2,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.78},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.228}]},{"osd":3,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.335}]},{"osd":4,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.921}]},{"osd":5,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":2.093},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.853}]},{"osd":6,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":2.032},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":2.0129999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":160277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.526},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.6919999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.601}]},{"osd":3,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.288}]},{"osd":4,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.397},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.3420000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.6359999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.427},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.111}]}]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":160333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.2450000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.5449999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.175}]},{"osd":4,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.9119999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.3600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.0299999999999998,"5min":3.0299999999999998,"15min":3.0299999999999998},"last":1.7370000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.546,"15min":1.546},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.5840000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":159765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.6000000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.097},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.95999999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.353},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.671}]},{"osd":3,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.633}]},{"osd":5,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":4.3979999999999997,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":4.2060000000000004,"5min":4.2060000000000004,"15min":4.2060000000000004},"last":1.706}]},{"osd":6,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.0339999999999998,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.335,"5min":4.335,"15min":4.335},"last":1.9630000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":3.964,"5min":3.964,"15min":3.964},"last":2.016},{"interface":"front","average":{"1min":1.661,"5min":1.661,"15min":1.661},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.8809999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":1.9970000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.0669999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.214},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.389},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.298}]},{"osd":5,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.538},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.6319999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.502},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.663}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":159731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.83899999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.1779999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.996}]},{"osd":4,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.51},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.3839999999999999,"5min":3.3839999999999999,"15min":3.3839999999999999},"last":1.4450000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":1.4059999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.218},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.375}]},{"osd":7,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.9969999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":1.3540000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.032}]},{"osd":2,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.6870000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":1.615}]},{"osd":5,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.163},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.454}]},{"osd":6,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.52},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.5660000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.3999999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":750029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.119},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.02}]},{"osd":2,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.179},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.504}]},{"osd":5,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.333},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1}]},{"osd":6,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":3.3730000000000002,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.4039999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.3700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:15.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.013+0000 7fa6c8861640 1 Processor -- start 2023-11-24T06:29:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.013+0000 7fa6c8861640 1 -- start start 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c8861640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa6c40cd240 0x7fa6c40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c8861640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa6c409a170 0x7fa6c40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c8861640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c406cea0 con 0x7fa6c406c6b0 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c8861640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c406cbc0 con 0x7fa6c40cd240 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c8861640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c406f350 con 0x7fa6c409a170 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa6c40cd240 0x7fa6c40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c1d9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa6c406c6b0 0x7fa6c4099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59520/0 (socket says 172.21.15.64:59520) 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.014+0000 7fa6c1d9b640 1 -- 172.21.15.64:0/1776562265 learned_addr learned my addr 172.21.15.64:0/1776562265 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:16.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.015+0000 7fa6c2d9d640 1 --2- 172.21.15.64:0/1776562265 >> v2:172.21.15.162:3300/0 conn(0x7fa6c409a170 0x7fa6c40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.015+0000 7fa6c259c640 1 -- 172.21.15.64:0/1776562265 >> v2:172.21.15.162:3300/0 conn(0x7fa6c409a170 msgr2=0x7fa6c40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.015+0000 7fa6c259c640 1 --2- 172.21.15.64:0/1776562265 >> v2:172.21.15.162:3300/0 conn(0x7fa6c409a170 0x7fa6c40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.015+0000 7fa6c259c640 1 -- 172.21.15.64:0/1776562265 >> v1:172.21.15.64:6789/0 conn(0x7fa6c406c6b0 legacy=0x7fa6c4099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.015+0000 7fa6c259c640 1 -- 172.21.15.64:0/1776562265 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c406f630 con 0x7fa6c40cd240 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.015+0000 7fa6c159a640 1 -- 172.21.15.64:0/1776562265 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970569791 0 0) 0x7fa6c406cea0 con 0x7fa6c406c6b0 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.016+0000 7fa6c259c640 1 --2- 172.21.15.64:0/1776562265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c40cd240 0x7fa6c40b3820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cb7f0 tx=0x7fa6b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f798ede753f60b87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.016+0000 7fa6c159a640 1 -- 172.21.15.64:0/1776562265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4031830 con 0x7fa6c40cd240 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.016+0000 7fa6c159a640 1 -- 172.21.15.64:0/1776562265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4031990 con 0x7fa6c40cd240 2023-11-24T06:29:16.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.016+0000 7fa6c159a640 1 -- 172.21.15.64:0/1776562265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4031d20 con 0x7fa6c40cd240 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 -- 172.21.15.64:0/1776562265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c40cd240 msgr2=0x7fa6c40b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 --2- 172.21.15.64:0/1776562265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c40cd240 0x7fa6c40b3820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cb7f0 tx=0x7fa6b402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 -- 172.21.15.64:0/1776562265 shutdown_connections 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 --2- 172.21.15.64:0/1776562265 >> v2:172.21.15.162:3300/0 conn(0x7fa6c409a170 0x7fa6c40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 --2- 172.21.15.64:0/1776562265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c40cd240 0x7fa6c40b3820 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 -- 172.21.15.64:0/1776562265 >> 172.21.15.64:0/1776562265 conn(0x7fa6c40b2d00 msgr2=0x7fa6c40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 -- 172.21.15.64:0/1776562265 shutdown_connections 2023-11-24T06:29:16.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.017+0000 7fa6c8861640 1 -- 172.21.15.64:0/1776562265 wait complete. 2023-11-24T06:29:16.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.019+0000 7fa6c8861640 1 Processor -- start 2023-11-24T06:29:16.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.019+0000 7fa6c8861640 1 -- start start 2023-11-24T06:29:16.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c8861640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c406c6b0 0x7fa6c4157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c8861640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c409a170 0x7fa6c40c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c8861640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c40cd240 0x7fa6c415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c8861640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41f2ce0 con 0x7fa6c409a170 2023-11-24T06:29:16.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c8861640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40c57a0 con 0x7fa6c406c6b0 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c8861640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40c5a80 con 0x7fa6c40cd240 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c409a170 0x7fa6c40c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c409a170 0x7fa6c40c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37360/0 (socket says 172.21.15.64:37360) 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.020+0000 7fa6c1d9b640 1 -- 172.21.15.64:0/142568208 learned_addr learned my addr 172.21.15.64:0/142568208 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c259c640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c406c6b0 0x7fa6c4157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c1d9b640 1 -- 172.21.15.64:0/142568208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c40cd240 msgr2=0x7fa6c415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c1d9b640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c40cd240 0x7fa6c415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c1d9b640 1 -- 172.21.15.64:0/142568208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c406c6b0 msgr2=0x7fa6c4157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c1d9b640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c406c6b0 0x7fa6c4157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c1d9b640 1 -- 172.21.15.64:0/142568208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4009400 con 0x7fa6c409a170 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c1d9b640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c409a170 0x7fa6c40c4d00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c4b0 tx=0x7fa6b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/142568208 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80074e0 con 0x7fa6c409a170 2023-11-24T06:29:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/142568208 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b8007660 con 0x7fa6c409a170 2023-11-24T06:29:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/142568208 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8004a20 con 0x7fa6c409a170 2023-11-24T06:29:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.021+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40c5d60 con 0x7fa6c409a170 2023-11-24T06:29:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.022+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40c6320 con 0x7fa6c409a170 2023-11-24T06:29:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.023+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c40c6540 con 0x7fa6c409a170 2023-11-24T06:29:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.024+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/142568208 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6b8004b80 con 0x7fa6c409a170 2023-11-24T06:29:16.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.025+0000 7fa6b2ffd640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa6a406ae40 0x7fa6a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.025+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/142568208 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6b8088800 con 0x7fa6c409a170 2023-11-24T06:29:16.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.025+0000 7fa6c259c640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa6a406ae40 0x7fa6a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.026+0000 7fa6c259c640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa6a406ae40 0x7fa6a406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4036000 tx=0x7fa6b403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.029+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/142568208 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6b80538e0 con 0x7fa6c409a170 2023-11-24T06:29:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.218+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6c40cd620 con 0x7fa6a406ae40 2023-11-24T06:29:16.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.227+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/142568208 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286313 (secure 0 0 0) 0x7fa6b4044400 con 0x7fa6a406ae40 2023-11-24T06:29:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:16.235 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:29:16.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa6a406ae40 msgr2=0x7fa6a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa6a406ae40 0x7fa6a406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4036000 tx=0x7fa6b403c040 comp rx=0 tx=0).stop 2023-11-24T06:29:16.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c409a170 msgr2=0x7fa6c40c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c409a170 0x7fa6c40c4d00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c4b0 tx=0x7fa6b800c970 comp rx=0 tx=0).stop 2023-11-24T06:29:16.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 shutdown_connections 2023-11-24T06:29:16.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa6a406ae40 0x7fa6a406d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c40cd240 0x7fa6c415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c409a170 0x7fa6c40c4d00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 --2- 172.21.15.64:0/142568208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c406c6b0 0x7fa6c4157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.232+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 >> 172.21.15.64:0/142568208 conn(0x7fa6c40b2d00 msgr2=0x7fa6c40b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:16.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.233+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 shutdown_connections 2023-11-24T06:29:16.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.233+0000 7fa6c8861640 1 -- 172.21.15.64:0/142568208 wait complete. 2023-11-24T06:29:16.256 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-24T06:29:15.128247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220564,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754156,"statfs":{"total":773094113280,"available":772868255744,"internally_reserved":0,"allocated":4497408,"data_stored":2462844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.897278"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681781+0000","last_change":"2023-11-24T06:29:11.282621+0000","last_active":"2023-11-24T06:29:12.681781+0000","last_peered":"2023-11-24T06:29:12.681781+0000","last_clean":"2023-11-24T06:29:12.681781+0000","last_became_active":"2023-11-24T06:29:11.282310+0000","last_became_peered":"2023-11-24T06:29:11.282310+0000","last_unstale":"2023-11-24T06:29:12.681781+0000","last_undegraded":"2023-11-24T06:29:12.681781+0000","last_fullsized":"2023-11-24T06:29:12.681781+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:47.628375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683470+0000","last_change":"2023-11-24T06:29:11.311148+0000","last_active":"2023-11-24T06:29:12.683470+0000","last_peered":"2023-11-24T06:29:12.683470+0000","last_clean":"2023-11-24T06:29:12.683470+0000","last_became_active":"2023-11-24T06:29:11.308037+0000","last_became_peered":"2023-11-24T06:29:11.308037+0000","last_unstale":"2023-11-24T06:29:12.683470+0000","last_undegraded":"2023-11-24T06:29:12.683470+0000","last_fullsized":"2023-11-24T06:29:12.683470+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:02.393482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290475+0000","last_change":"2023-11-24T06:29:11.293230+0000","last_active":"2023-11-24T06:29:12.290475+0000","last_peered":"2023-11-24T06:29:12.290475+0000","last_clean":"2023-11-24T06:29:12.290475+0000","last_became_active":"2023-11-24T06:29:11.292929+0000","last_became_peered":"2023-11-24T06:29:11.292929+0000","last_unstale":"2023-11-24T06:29:12.290475+0000","last_undegraded":"2023-11-24T06:29:12.290475+0000","last_fullsized":"2023-11-24T06:29:12.290475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:24.991791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681028+0000","last_change":"2023-11-24T06:29:11.287940+0000","last_active":"2023-11-24T06:29:12.681028+0000","last_peered":"2023-11-24T06:29:12.681028+0000","last_clean":"2023-11-24T06:29:12.681028+0000","last_became_active":"2023-11-24T06:29:11.287627+0000","last_became_peered":"2023-11-24T06:29:11.287627+0000","last_unstale":"2023-11-24T06:29:12.681028+0000","last_undegraded":"2023-11-24T06:29:12.681028+0000","last_fullsized":"2023-11-24T06:29:12.681028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:15.117892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285485+0000","last_change":"2023-11-24T06:29:11.309984+0000","last_active":"2023-11-24T06:29:12.285485+0000","last_peered":"2023-11-24T06:29:12.285485+0000","last_clean":"2023-11-24T06:29:12.285485+0000","last_became_active":"2023-11-24T06:29:11.309562+0000","last_became_peered":"2023-11-24T06:29:11.309562+0000","last_unstale":"2023-11-24T06:29:12.285485+0000","last_undegraded":"2023-11-24T06:29:12.285485+0000","last_fullsized":"2023-11-24T06:29:12.285485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:57.411653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681528+0000","last_change":"2023-11-24T06:29:11.308671+0000","last_active":"2023-11-24T06:29:12.681528+0000","last_peered":"2023-11-24T06:29:12.681528+0000","last_clean":"2023-11-24T06:29:12.681528+0000","last_became_active":"2023-11-24T06:29:11.308342+0000","last_became_peered":"2023-11-24T06:29:11.308342+0000","last_unstale":"2023-11-24T06:29:12.681528+0000","last_undegraded":"2023-11-24T06:29:12.681528+0000","last_fullsized":"2023-11-24T06:29:12.681528+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:56.242991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328062+0000","last_change":"2023-11-24T06:29:11.287205+0000","last_active":"2023-11-24T06:29:12.328062+0000","last_peered":"2023-11-24T06:29:12.328062+0000","last_clean":"2023-11-24T06:29:12.328062+0000","last_became_active":"2023-11-24T06:29:11.285679+0000","last_became_peered":"2023-11-24T06:29:11.285679+0000","last_unstale":"2023-11-24T06:29:12.328062+0000","last_undegraded":"2023-11-24T06:29:12.328062+0000","last_fullsized":"2023-11-24T06:29:12.328062+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:30.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327290+0000","last_change":"2023-11-24T06:29:11.276395+0000","last_active":"2023-11-24T06:29:12.327290+0000","last_peered":"2023-11-24T06:29:12.327290+0000","last_clean":"2023-11-24T06:29:12.327290+0000","last_became_active":"2023-11-24T06:29:11.275962+0000","last_became_peered":"2023-11-24T06:29:11.275962+0000","last_unstale":"2023-11-24T06:29:12.327290+0000","last_undegraded":"2023-11-24T06:29:12.327290+0000","last_fullsized":"2023-11-24T06:29:12.327290+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:36.625390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289719+0000","last_change":"2023-11-24T06:29:11.281288+0000","last_active":"2023-11-24T06:29:12.289719+0000","last_peered":"2023-11-24T06:29:12.289719+0000","last_clean":"2023-11-24T06:29:12.289719+0000","last_became_active":"2023-11-24T06:29:11.280975+0000","last_became_peered":"2023-11-24T06:29:11.280975+0000","last_unstale":"2023-11-24T06:29:12.289719+0000","last_undegraded":"2023-11-24T06:29:12.289719+0000","last_fullsized":"2023-11-24T06:29:12.289719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:24.991791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681158+0000","last_change":"2023-11-24T06:29:11.311472+0000","last_active":"2023-11-24T06:29:12.681158+0000","last_peered":"2023-11-24T06:29:12.681158+0000","last_clean":"2023-11-24T06:29:12.681158+0000","last_became_active":"2023-11-24T06:29:11.311169+0000","last_became_peered":"2023-11-24T06:29:11.311169+0000","last_unstale":"2023-11-24T06:29:12.681158+0000","last_undegraded":"2023-11-24T06:29:12.681158+0000","last_fullsized":"2023-11-24T06:29:12.681158+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:15:34.296527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328282+0000","last_change":"2023-11-24T06:29:11.288305+0000","last_active":"2023-11-24T06:29:12.328282+0000","last_peered":"2023-11-24T06:29:12.328282+0000","last_clean":"2023-11-24T06:29:12.328282+0000","last_became_active":"2023-11-24T06:29:11.288019+0000","last_became_peered":"2023-11-24T06:29:11.288019+0000","last_unstale":"2023-11-24T06:29:12.328282+0000","last_undegraded":"2023-11-24T06:29:12.328282+0000","last_fullsized":"2023-11-24T06:29:12.328282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:48.087034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290656+0000","last_change":"2023-11-24T06:29:11.311271+0000","last_active":"2023-11-24T06:29:12.290656+0000","last_peered":"2023-11-24T06:29:12.290656+0000","last_clean":"2023-11-24T06:29:12.290656+0000","last_became_active":"2023-11-24T06:29:11.310087+0000","last_became_peered":"2023-11-24T06:29:11.310087+0000","last_unstale":"2023-11-24T06:29:12.290656+0000","last_undegraded":"2023-11-24T06:29:12.290656+0000","last_fullsized":"2023-11-24T06:29:12.290656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:18.092054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286020+0000","last_change":"2023-11-24T06:29:11.282400+0000","last_active":"2023-11-24T06:29:12.286020+0000","last_peered":"2023-11-24T06:29:12.286020+0000","last_clean":"2023-11-24T06:29:12.286020+0000","last_became_active":"2023-11-24T06:29:11.280456+0000","last_became_peered":"2023-11-24T06:29:11.280456+0000","last_unstale":"2023-11-24T06:29:12.286020+0000","last_undegraded":"2023-11-24T06:29:12.286020+0000","last_fullsized":"2023-11-24T06:29:12.286020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:07:23.526675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681698+0000","last_change":"2023-11-24T06:29:11.310568+0000","last_active":"2023-11-24T06:29:12.681698+0000","last_peered":"2023-11-24T06:29:12.681698+0000","last_clean":"2023-11-24T06:29:12.681698+0000","last_became_active":"2023-11-24T06:29:11.307987+0000","last_became_peered":"2023-11-24T06:29:11.307987+0000","last_unstale":"2023-11-24T06:29:12.681698+0000","last_undegraded":"2023-11-24T06:29:12.681698+0000","last_fullsized":"2023-11-24T06:29:12.681698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:00.524180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289558+0000","last_change":"2023-11-24T06:29:11.288420+0000","last_active":"2023-11-24T06:29:12.289558+0000","last_peered":"2023-11-24T06:29:12.289558+0000","last_clean":"2023-11-24T06:29:12.289558+0000","last_became_active":"2023-11-24T06:29:11.286993+0000","last_became_peered":"2023-11-24T06:29:11.286993+0000","last_unstale":"2023-11-24T06:29:12.289558+0000","last_undegraded":"2023-11-24T06:29:12.289558+0000","last_fullsized":"2023-11-24T06:29:12.289558+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:29.315454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328224+0000","last_change":"2023-11-24T06:29:11.314102+0000","last_active":"2023-11-24T06:29:12.328224+0000","last_peered":"2023-11-24T06:29:12.328224+0000","last_clean":"2023-11-24T06:29:12.328224+0000","last_became_active":"2023-11-24T06:29:11.313762+0000","last_became_peered":"2023-11-24T06:29:11.313762+0000","last_unstale":"2023-11-24T06:29:12.328224+0000","last_undegraded":"2023-11-24T06:29:12.328224+0000","last_fullsized":"2023-11-24T06:29:12.328224+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:18.092054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328557+0000","last_change":"2023-11-24T06:29:11.290508+0000","last_active":"2023-11-24T06:29:12.328557+0000","last_peered":"2023-11-24T06:29:12.328557+0000","last_clean":"2023-11-24T06:29:12.328557+0000","last_became_active":"2023-11-24T06:29:11.290238+0000","last_became_peered":"2023-11-24T06:29:11.290238+0000","last_unstale":"2023-11-24T06:29:12.328557+0000","last_undegraded":"2023-11-24T06:29:12.328557+0000","last_fullsized":"2023-11-24T06:29:12.328557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:06.364499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683598+0000","last_change":"2023-11-24T06:29:11.301367+0000","last_active":"2023-11-24T06:29:12.683598+0000","last_peered":"2023-11-24T06:29:12.683598+0000","last_clean":"2023-11-24T06:29:12.683598+0000","last_became_active":"2023-11-24T06:29:11.300442+0000","last_became_peered":"2023-11-24T06:29:11.300442+0000","last_unstale":"2023-11-24T06:29:12.683598+0000","last_undegraded":"2023-11-24T06:29:12.683598+0000","last_fullsized":"2023-11-24T06:29:12.683598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:26:28.634476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683234+0000","last_change":"2023-11-24T06:29:11.301305+0000","last_active":"2023-11-24T06:29:12.683234+0000","last_peered":"2023-11-24T06:29:12.683234+0000","last_clean":"2023-11-24T06:29:12.683234+0000","last_became_active":"2023-11-24T06:29:11.300562+0000","last_became_peered":"2023-11-24T06:29:11.300562+0000","last_unstale":"2023-11-24T06:29:12.683234+0000","last_undegraded":"2023-11-24T06:29:12.683234+0000","last_fullsized":"2023-11-24T06:29:12.683234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:00.659984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285928+0000","last_change":"2023-11-24T06:29:11.288291+0000","last_active":"2023-11-24T06:29:12.285928+0000","last_peered":"2023-11-24T06:29:12.285928+0000","last_clean":"2023-11-24T06:29:12.285928+0000","last_became_active":"2023-11-24T06:29:11.287949+0000","last_became_peered":"2023-11-24T06:29:11.287949+0000","last_unstale":"2023-11-24T06:29:12.285928+0000","last_undegraded":"2023-11-24T06:29:12.285928+0000","last_fullsized":"2023-11-24T06:29:12.285928+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:10.213036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288479+0000","last_change":"2023-11-24T06:29:11.291383+0000","last_active":"2023-11-24T06:29:12.288479+0000","last_peered":"2023-11-24T06:29:12.288479+0000","last_clean":"2023-11-24T06:29:12.288479+0000","last_became_active":"2023-11-24T06:29:11.288825+0000","last_became_peered":"2023-11-24T06:29:11.288825+0000","last_unstale":"2023-11-24T06:29:12.288479+0000","last_undegraded":"2023-11-24T06:29:12.288479+0000","last_fullsized":"2023-11-24T06:29:12.288479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:13.161846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289203+0000","last_change":"2023-11-24T06:29:11.310994+0000","last_active":"2023-11-24T06:29:12.289203+0000","last_peered":"2023-11-24T06:29:12.289203+0000","last_clean":"2023-11-24T06:29:12.289203+0000","last_became_active":"2023-11-24T06:29:11.310647+0000","last_became_peered":"2023-11-24T06:29:11.310647+0000","last_unstale":"2023-11-24T06:29:12.289203+0000","last_undegraded":"2023-11-24T06:29:12.289203+0000","last_fullsized":"2023-11-24T06:29:12.289203+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:29.315454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681914+0000","last_change":"2023-11-24T06:29:11.285669+0000","last_active":"2023-11-24T06:29:12.681914+0000","last_peered":"2023-11-24T06:29:12.681914+0000","last_clean":"2023-11-24T06:29:12.681914+0000","last_became_active":"2023-11-24T06:29:11.285388+0000","last_became_peered":"2023-11-24T06:29:11.285388+0000","last_unstale":"2023-11-24T06:29:12.681914+0000","last_undegraded":"2023-11-24T06:29:12.681914+0000","last_fullsized":"2023-11-24T06:29:12.681914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:47.628375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288824+0000","last_change":"2023-11-24T06:29:11.286815+0000","last_active":"2023-11-24T06:29:12.288824+0000","last_peered":"2023-11-24T06:29:12.288824+0000","last_clean":"2023-11-24T06:29:12.288824+0000","last_became_active":"2023-11-24T06:29:11.286466+0000","last_became_peered":"2023-11-24T06:29:11.286466+0000","last_unstale":"2023-11-24T06:29:12.288824+0000","last_undegraded":"2023-11-24T06:29:12.288824+0000","last_fullsized":"2023-11-24T06:29:12.288824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:14:19.485966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681218+0000","last_change":"2023-11-24T06:29:11.280784+0000","last_active":"2023-11-24T06:29:12.681218+0000","last_peered":"2023-11-24T06:29:12.681218+0000","last_clean":"2023-11-24T06:29:12.681218+0000","last_became_active":"2023-11-24T06:29:11.280144+0000","last_became_peered":"2023-11-24T06:29:11.280144+0000","last_unstale":"2023-11-24T06:29:12.681218+0000","last_undegraded":"2023-11-24T06:29:12.681218+0000","last_fullsized":"2023-11-24T06:29:12.681218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:19.766174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683947+0000","last_change":"2023-11-24T06:29:11.300033+0000","last_active":"2023-11-24T06:29:12.683947+0000","last_peered":"2023-11-24T06:29:12.683947+0000","last_clean":"2023-11-24T06:29:12.683947+0000","last_became_active":"2023-11-24T06:29:11.299009+0000","last_became_peered":"2023-11-24T06:29:11.299009+0000","last_unstale":"2023-11-24T06:29:12.683947+0000","last_undegraded":"2023-11-24T06:29:12.683947+0000","last_fullsized":"2023-11-24T06:29:12.683947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:48.761991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681879+0000","last_change":"2023-11-24T06:29:11.292692+0000","last_active":"2023-11-24T06:29:12.681879+0000","last_peered":"2023-11-24T06:29:12.681879+0000","last_clean":"2023-11-24T06:29:12.681879+0000","last_became_active":"2023-11-24T06:29:11.292407+0000","last_became_peered":"2023-11-24T06:29:11.292407+0000","last_unstale":"2023-11-24T06:29:12.681879+0000","last_undegraded":"2023-11-24T06:29:12.681879+0000","last_fullsized":"2023-11-24T06:29:12.681879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:24.044724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290490+0000","last_change":"2023-11-24T06:29:11.289281+0000","last_active":"2023-11-24T06:29:12.290490+0000","last_peered":"2023-11-24T06:29:12.290490+0000","last_clean":"2023-11-24T06:29:12.290490+0000","last_became_active":"2023-11-24T06:29:11.288986+0000","last_became_peered":"2023-11-24T06:29:11.288986+0000","last_unstale":"2023-11-24T06:29:12.290490+0000","last_undegraded":"2023-11-24T06:29:12.290490+0000","last_fullsized":"2023-11-24T06:29:12.290490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:30.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288898+0000","last_change":"2023-11-24T06:29:11.310739+0000","last_active":"2023-11-24T06:29:12.288898+0000","last_peered":"2023-11-24T06:29:12.288898+0000","last_clean":"2023-11-24T06:29:12.288898+0000","last_became_active":"2023-11-24T06:29:11.310411+0000","last_became_peered":"2023-11-24T06:29:11.310411+0000","last_unstale":"2023-11-24T06:29:12.288898+0000","last_undegraded":"2023-11-24T06:29:12.288898+0000","last_fullsized":"2023-11-24T06:29:12.288898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:14:19.485966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327706+0000","last_change":"2023-11-24T06:29:11.284502+0000","last_active":"2023-11-24T06:29:12.327706+0000","last_peered":"2023-11-24T06:29:12.327706+0000","last_clean":"2023-11-24T06:29:12.327706+0000","last_became_active":"2023-11-24T06:29:11.283395+0000","last_became_peered":"2023-11-24T06:29:11.283395+0000","last_unstale":"2023-11-24T06:29:12.327706+0000","last_undegraded":"2023-11-24T06:29:12.327706+0000","last_fullsized":"2023-11-24T06:29:12.327706+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:38.902004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681367+0000","last_change":"2023-11-24T06:29:11.283330+0000","last_active":"2023-11-24T06:29:12.681367+0000","last_peered":"2023-11-24T06:29:12.681367+0000","last_clean":"2023-11-24T06:29:12.681367+0000","last_became_active":"2023-11-24T06:29:11.283039+0000","last_became_peered":"2023-11-24T06:29:11.283039+0000","last_unstale":"2023-11-24T06:29:12.681367+0000","last_undegraded":"2023-11-24T06:29:12.681367+0000","last_fullsized":"2023-11-24T06:29:12.681367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:38.862116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327402+0000","last_change":"2023-11-24T06:29:11.285666+0000","last_active":"2023-11-24T06:29:12.327402+0000","last_peered":"2023-11-24T06:29:12.327402+0000","last_clean":"2023-11-24T06:29:12.327402+0000","last_became_active":"2023-11-24T06:29:11.282590+0000","last_became_peered":"2023-11-24T06:29:11.282590+0000","last_unstale":"2023-11-24T06:29:12.327402+0000","last_undegraded":"2023-11-24T06:29:12.327402+0000","last_fullsized":"2023-11-24T06:29:12.327402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:14:19.485966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682665+0000","last_change":"2023-11-24T06:29:11.290739+0000","last_active":"2023-11-24T06:29:12.682665+0000","last_peered":"2023-11-24T06:29:12.682665+0000","last_clean":"2023-11-24T06:29:12.682665+0000","last_became_active":"2023-11-24T06:29:11.281388+0000","last_became_peered":"2023-11-24T06:29:11.281388+0000","last_unstale":"2023-11-24T06:29:12.682665+0000","last_undegraded":"2023-11-24T06:29:12.682665+0000","last_fullsized":"2023-11-24T06:29:12.682665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:49:20.918194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681915+0000","last_change":"2023-11-24T06:29:10.281226+0000","last_active":"2023-11-24T06:29:12.681915+0000","last_peered":"2023-11-24T06:29:12.681915+0000","last_clean":"2023-11-24T06:29:12.681915+0000","last_became_active":"2023-11-24T06:29:10.280942+0000","last_became_peered":"2023-11-24T06:29:10.280942+0000","last_unstale":"2023-11-24T06:29:12.681915+0000","last_undegraded":"2023-11-24T06:29:12.681915+0000","last_fullsized":"2023-11-24T06:29:12.681915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:07:22.512103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681886+0000","last_change":"2023-11-24T06:29:11.285786+0000","last_active":"2023-11-24T06:29:12.681886+0000","last_peered":"2023-11-24T06:29:12.681886+0000","last_clean":"2023-11-24T06:29:12.681886+0000","last_became_active":"2023-11-24T06:29:11.284070+0000","last_became_peered":"2023-11-24T06:29:11.284070+0000","last_unstale":"2023-11-24T06:29:12.681886+0000","last_undegraded":"2023-11-24T06:29:12.681886+0000","last_fullsized":"2023-11-24T06:29:12.681886+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:52.500523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682794+0000","last_change":"2023-11-24T06:29:10.282192+0000","last_active":"2023-11-24T06:29:12.682794+0000","last_peered":"2023-11-24T06:29:12.682794+0000","last_clean":"2023-11-24T06:29:12.682794+0000","last_became_active":"2023-11-24T06:29:10.281906+0000","last_became_peered":"2023-11-24T06:29:10.281906+0000","last_unstale":"2023-11-24T06:29:12.682794+0000","last_undegraded":"2023-11-24T06:29:12.682794+0000","last_fullsized":"2023-11-24T06:29:12.682794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:30.133018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289478+0000","last_change":"2023-11-24T06:29:11.310741+0000","last_active":"2023-11-24T06:29:12.289478+0000","last_peered":"2023-11-24T06:29:12.289478+0000","last_clean":"2023-11-24T06:29:12.289478+0000","last_became_active":"2023-11-24T06:29:11.310445+0000","last_became_peered":"2023-11-24T06:29:11.310445+0000","last_unstale":"2023-11-24T06:29:12.289478+0000","last_undegraded":"2023-11-24T06:29:12.289478+0000","last_fullsized":"2023-11-24T06:29:12.289478+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:38.902004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683413+0000","last_change":"2023-11-24T06:29:10.283612+0000","last_active":"2023-11-24T06:29:12.683413+0000","last_peered":"2023-11-24T06:29:12.683413+0000","last_clean":"2023-11-24T06:29:12.683413+0000","last_became_active":"2023-11-24T06:29:10.283281+0000","last_became_peered":"2023-11-24T06:29:10.283281+0000","last_unstale":"2023-11-24T06:29:12.683413+0000","last_undegraded":"2023-11-24T06:29:12.683413+0000","last_fullsized":"2023-11-24T06:29:12.683413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:53.896209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682225+0000","last_change":"2023-11-24T06:29:11.285921+0000","last_active":"2023-11-24T06:29:12.682225+0000","last_peered":"2023-11-24T06:29:12.682225+0000","last_clean":"2023-11-24T06:29:12.682225+0000","last_became_active":"2023-11-24T06:29:11.285457+0000","last_became_peered":"2023-11-24T06:29:11.285457+0000","last_unstale":"2023-11-24T06:29:12.682225+0000","last_undegraded":"2023-11-24T06:29:12.682225+0000","last_fullsized":"2023-11-24T06:29:12.682225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:25.204869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286065+0000","last_change":"2023-11-24T06:29:10.273947+0000","last_active":"2023-11-24T06:29:12.286065+0000","last_peered":"2023-11-24T06:29:12.286065+0000","last_clean":"2023-11-24T06:29:12.286065+0000","last_became_active":"2023-11-24T06:29:10.273640+0000","last_became_peered":"2023-11-24T06:29:10.273640+0000","last_unstale":"2023-11-24T06:29:12.286065+0000","last_undegraded":"2023-11-24T06:29:12.286065+0000","last_fullsized":"2023-11-24T06:29:12.286065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:43.191205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.328361+0000","last_change":"2023-11-24T06:29:11.281017+0000","last_active":"2023-11-24T06:29:12.328361+0000","last_peered":"2023-11-24T06:29:12.328361+0000","last_clean":"2023-11-24T06:29:12.328361+0000","last_became_active":"2023-11-24T06:29:11.280434+0000","last_became_peered":"2023-11-24T06:29:11.280434+0000","last_unstale":"2023-11-24T06:29:12.328361+0000","last_undegraded":"2023-11-24T06:29:12.328361+0000","last_fullsized":"2023-11-24T06:29:12.328361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:10.384116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290254+0000","last_change":"2023-11-24T06:29:10.278551+0000","last_active":"2023-11-24T06:29:12.290254+0000","last_peered":"2023-11-24T06:29:12.290254+0000","last_clean":"2023-11-24T06:29:12.290254+0000","last_became_active":"2023-11-24T06:29:10.278253+0000","last_became_peered":"2023-11-24T06:29:10.278253+0000","last_unstale":"2023-11-24T06:29:12.290254+0000","last_undegraded":"2023-11-24T06:29:12.290254+0000","last_fullsized":"2023-11-24T06:29:12.290254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:56.397081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290282+0000","last_change":"2023-11-24T06:29:11.295838+0000","last_active":"2023-11-24T06:29:12.290282+0000","last_peered":"2023-11-24T06:29:12.290282+0000","last_clean":"2023-11-24T06:29:12.290282+0000","last_became_active":"2023-11-24T06:29:11.295247+0000","last_became_peered":"2023-11-24T06:29:11.295247+0000","last_unstale":"2023-11-24T06:29:12.290282+0000","last_undegraded":"2023-11-24T06:29:12.290282+0000","last_fullsized":"2023-11-24T06:29:12.290282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:14.532797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289443+0000","last_change":"2023-11-24T06:29:10.284802+0000","last_active":"2023-11-24T06:29:12.289443+0000","last_peered":"2023-11-24T06:29:12.289443+0000","last_clean":"2023-11-24T06:29:12.289443+0000","last_became_active":"2023-11-24T06:29:10.283754+0000","last_became_peered":"2023-11-24T06:29:10.283754+0000","last_unstale":"2023-11-24T06:29:12.289443+0000","last_undegraded":"2023-11-24T06:29:12.289443+0000","last_fullsized":"2023-11-24T06:29:12.289443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:09:59.719383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288765+0000","last_change":"2023-11-24T06:29:11.285574+0000","last_active":"2023-11-24T06:29:12.288765+0000","last_peered":"2023-11-24T06:29:12.288765+0000","last_clean":"2023-11-24T06:29:12.288765+0000","last_became_active":"2023-11-24T06:29:11.285257+0000","last_became_peered":"2023-11-24T06:29:11.285257+0000","last_unstale":"2023-11-24T06:29:12.288765+0000","last_undegraded":"2023-11-24T06:29:12.288765+0000","last_fullsized":"2023-11-24T06:29:12.288765+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:36.625390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680887+0000","last_change":"2023-11-24T06:29:10.265611+0000","last_active":"2023-11-24T06:29:12.680887+0000","last_peered":"2023-11-24T06:29:12.680887+0000","last_clean":"2023-11-24T06:29:12.680887+0000","last_became_active":"2023-11-24T06:29:10.265275+0000","last_became_peered":"2023-11-24T06:29:10.265275+0000","last_unstale":"2023-11-24T06:29:12.680887+0000","last_undegraded":"2023-11-24T06:29:12.680887+0000","last_fullsized":"2023-11-24T06:29:12.680887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:04.116234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680637+0000","last_change":"2023-11-24T06:29:11.282332+0000","last_active":"2023-11-24T06:29:12.680637+0000","last_peered":"2023-11-24T06:29:12.680637+0000","last_clean":"2023-11-24T06:29:12.680637+0000","last_became_active":"2023-11-24T06:29:11.282064+0000","last_became_peered":"2023-11-24T06:29:11.282064+0000","last_unstale":"2023-11-24T06:29:12.680637+0000","last_undegraded":"2023-11-24T06:29:12.680637+0000","last_fullsized":"2023-11-24T06:29:12.680637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:57.678666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288307+0000","last_change":"2023-11-24T06:29:10.276033+0000","last_active":"2023-11-24T06:29:12.288307+0000","last_peered":"2023-11-24T06:29:12.288307+0000","last_clean":"2023-11-24T06:29:12.288307+0000","last_became_active":"2023-11-24T06:29:10.275731+0000","last_became_peered":"2023-11-24T06:29:10.275731+0000","last_unstale":"2023-11-24T06:29:12.288307+0000","last_undegraded":"2023-11-24T06:29:12.288307+0000","last_fullsized":"2023-11-24T06:29:12.288307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:05.645745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.291198+0000","last_change":"2023-11-24T06:29:11.310394+0000","last_active":"2023-11-24T06:29:12.291198+0000","last_peered":"2023-11-24T06:29:12.291198+0000","last_clean":"2023-11-24T06:29:12.291198+0000","last_became_active":"2023-11-24T06:29:11.309916+0000","last_became_peered":"2023-11-24T06:29:11.309916+0000","last_unstale":"2023-11-24T06:29:12.291198+0000","last_undegraded":"2023-11-24T06:29:12.291198+0000","last_fullsized":"2023-11-24T06:29:12.291198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:10.384116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683678+0000","last_change":"2023-11-24T06:29:10.284021+0000","last_active":"2023-11-24T06:29:12.683678+0000","last_peered":"2023-11-24T06:29:12.683678+0000","last_clean":"2023-11-24T06:29:12.683678+0000","last_became_active":"2023-11-24T06:29:10.283345+0000","last_became_peered":"2023-11-24T06:29:10.283345+0000","last_unstale":"2023-11-24T06:29:12.683678+0000","last_undegraded":"2023-11-24T06:29:12.683678+0000","last_fullsized":"2023-11-24T06:29:12.683678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:42.009905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683567+0000","last_change":"2023-11-24T06:29:11.300344+0000","last_active":"2023-11-24T06:29:12.683567+0000","last_peered":"2023-11-24T06:29:12.683567+0000","last_clean":"2023-11-24T06:29:12.683567+0000","last_became_active":"2023-11-24T06:29:11.299842+0000","last_became_peered":"2023-11-24T06:29:11.299842+0000","last_unstale":"2023-11-24T06:29:12.683567+0000","last_undegraded":"2023-11-24T06:29:12.683567+0000","last_fullsized":"2023-11-24T06:29:12.683567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:14.542488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682715+0000","last_change":"2023-11-24T06:29:10.269271+0000","last_active":"2023-11-24T06:29:12.682715+0000","last_peered":"2023-11-24T06:29:12.682715+0000","last_clean":"2023-11-24T06:29:12.682715+0000","last_became_active":"2023-11-24T06:29:10.269002+0000","last_became_peered":"2023-11-24T06:29:10.269002+0000","last_unstale":"2023-11-24T06:29:12.682715+0000","last_undegraded":"2023-11-24T06:29:12.682715+0000","last_fullsized":"2023-11-24T06:29:12.682715+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:43.160017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683085+0000","last_change":"2023-11-24T06:29:11.308920+0000","last_active":"2023-11-24T06:29:12.683085+0000","last_peered":"2023-11-24T06:29:12.683085+0000","last_clean":"2023-11-24T06:29:12.683085+0000","last_became_active":"2023-11-24T06:29:11.305891+0000","last_became_peered":"2023-11-24T06:29:11.305891+0000","last_unstale":"2023-11-24T06:29:12.683085+0000","last_undegraded":"2023-11-24T06:29:12.683085+0000","last_fullsized":"2023-11-24T06:29:12.683085+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:46.529579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290812+0000","last_change":"2023-11-24T06:29:10.284433+0000","last_active":"2023-11-24T06:29:12.290812+0000","last_peered":"2023-11-24T06:29:12.290812+0000","last_clean":"2023-11-24T06:29:12.290812+0000","last_became_active":"2023-11-24T06:29:10.282509+0000","last_became_peered":"2023-11-24T06:29:10.282509+0000","last_unstale":"2023-11-24T06:29:12.290812+0000","last_undegraded":"2023-11-24T06:29:12.290812+0000","last_fullsized":"2023-11-24T06:29:12.290812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:47.226931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.290475+0000","last_change":"2023-11-24T06:29:11.298859+0000","last_active":"2023-11-24T06:29:12.290475+0000","last_peered":"2023-11-24T06:29:12.290475+0000","last_clean":"2023-11-24T06:29:12.290475+0000","last_became_active":"2023-11-24T06:29:11.298366+0000","last_became_peered":"2023-11-24T06:29:11.298366+0000","last_unstale":"2023-11-24T06:29:12.290475+0000","last_undegraded":"2023-11-24T06:29:12.290475+0000","last_fullsized":"2023-11-24T06:29:12.290475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:38.902004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.683162+0000","last_change":"2023-11-24T06:29:10.283604+0000","last_active":"2023-11-24T06:29:12.683162+0000","last_peered":"2023-11-24T06:29:12.683162+0000","last_clean":"2023-11-24T06:29:12.683162+0000","last_became_active":"2023-11-24T06:29:10.283259+0000","last_became_peered":"2023-11-24T06:29:10.283259+0000","last_unstale":"2023-11-24T06:29:12.683162+0000","last_undegraded":"2023-11-24T06:29:12.683162+0000","last_fullsized":"2023-11-24T06:29:12.683162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:14.103320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680475+0000","last_change":"2023-11-24T06:29:11.284438+0000","last_active":"2023-11-24T06:29:12.680475+0000","last_peered":"2023-11-24T06:29:12.680475+0000","last_clean":"2023-11-24T06:29:12.680475+0000","last_became_active":"2023-11-24T06:29:11.283701+0000","last_became_peered":"2023-11-24T06:29:11.283701+0000","last_unstale":"2023-11-24T06:29:12.680475+0000","last_undegraded":"2023-11-24T06:29:12.680475+0000","last_fullsized":"2023-11-24T06:29:12.680475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:18:43.024477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289236+0000","last_change":"2023-11-24T06:29:10.277155+0000","last_active":"2023-11-24T06:29:12.289236+0000","last_peered":"2023-11-24T06:29:12.289236+0000","last_clean":"2023-11-24T06:29:12.289236+0000","last_became_active":"2023-11-24T06:29:10.274961+0000","last_became_peered":"2023-11-24T06:29:10.274961+0000","last_unstale":"2023-11-24T06:29:12.289236+0000","last_undegraded":"2023-11-24T06:29:12.289236+0000","last_fullsized":"2023-11-24T06:29:12.289236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:30.133018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682175+0000","last_change":"2023-11-24T06:29:11.308955+0000","last_active":"2023-11-24T06:29:12.682175+0000","last_peered":"2023-11-24T06:29:12.682175+0000","last_clean":"2023-11-24T06:29:12.682175+0000","last_became_active":"2023-11-24T06:29:11.308662+0000","last_became_peered":"2023-11-24T06:29:11.308662+0000","last_unstale":"2023-11-24T06:29:12.682175+0000","last_undegraded":"2023-11-24T06:29:12.682175+0000","last_fullsized":"2023-11-24T06:29:12.682175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:12.393016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680573+0000","last_change":"2023-11-24T06:29:10.278376+0000","last_active":"2023-11-24T06:29:12.680573+0000","last_peered":"2023-11-24T06:29:12.680573+0000","last_clean":"2023-11-24T06:29:12.680573+0000","last_became_active":"2023-11-24T06:29:10.278051+0000","last_became_peered":"2023-11-24T06:29:10.278051+0000","last_unstale":"2023-11-24T06:29:12.680573+0000","last_undegraded":"2023-11-24T06:29:12.680573+0000","last_fullsized":"2023-11-24T06:29:12.680573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:36.148824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327704+0000","last_change":"2023-11-24T06:29:11.284085+0000","last_active":"2023-11-24T06:29:12.327704+0000","last_peered":"2023-11-24T06:29:12.327704+0000","last_clean":"2023-11-24T06:29:12.327704+0000","last_became_active":"2023-11-24T06:29:11.283771+0000","last_became_peered":"2023-11-24T06:29:11.283771+0000","last_unstale":"2023-11-24T06:29:12.327704+0000","last_undegraded":"2023-11-24T06:29:12.327704+0000","last_fullsized":"2023-11-24T06:29:12.327704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:14.532797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680893+0000","last_change":"2023-11-24T06:29:10.279929+0000","last_active":"2023-11-24T06:29:12.680893+0000","last_peered":"2023-11-24T06:29:12.680893+0000","last_clean":"2023-11-24T06:29:12.680893+0000","last_became_active":"2023-11-24T06:29:10.279575+0000","last_became_peered":"2023-11-24T06:29:10.279575+0000","last_unstale":"2023-11-24T06:29:12.680893+0000","last_undegraded":"2023-11-24T06:29:12.680893+0000","last_fullsized":"2023-11-24T06:29:12.680893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:42.651516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681482+0000","last_change":"2023-11-24T06:29:11.284874+0000","last_active":"2023-11-24T06:29:12.681482+0000","last_peered":"2023-11-24T06:29:12.681482+0000","last_clean":"2023-11-24T06:29:12.681482+0000","last_became_active":"2023-11-24T06:29:11.284391+0000","last_became_peered":"2023-11-24T06:29:11.284391+0000","last_unstale":"2023-11-24T06:29:12.681482+0000","last_undegraded":"2023-11-24T06:29:12.681482+0000","last_fullsized":"2023-11-24T06:29:12.681482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.216950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680419+0000","last_change":"2023-11-24T06:29:10.284791+0000","last_active":"2023-11-24T06:29:12.680419+0000","last_peered":"2023-11-24T06:29:12.680419+0000","last_clean":"2023-11-24T06:29:12.680419+0000","last_became_active":"2023-11-24T06:29:10.284485+0000","last_became_peered":"2023-11-24T06:29:10.284485+0000","last_unstale":"2023-11-24T06:29:12.680419+0000","last_undegraded":"2023-11-24T06:29:12.680419+0000","last_fullsized":"2023-11-24T06:29:12.680419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.751601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289326+0000","last_change":"2023-11-24T06:29:11.282035+0000","last_active":"2023-11-24T06:29:12.289326+0000","last_peered":"2023-11-24T06:29:12.289326+0000","last_clean":"2023-11-24T06:29:12.289326+0000","last_became_active":"2023-11-24T06:29:11.281762+0000","last_became_peered":"2023-11-24T06:29:11.281762+0000","last_unstale":"2023-11-24T06:29:12.289326+0000","last_undegraded":"2023-11-24T06:29:12.289326+0000","last_fullsized":"2023-11-24T06:29:12.289326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:14.532797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681363+0000","last_change":"2023-11-24T06:29:10.276069+0000","last_active":"2023-11-24T06:29:12.681363+0000","last_peered":"2023-11-24T06:29:12.681363+0000","last_clean":"2023-11-24T06:29:12.681363+0000","last_became_active":"2023-11-24T06:29:10.275767+0000","last_became_peered":"2023-11-24T06:29:10.275767+0000","last_unstale":"2023-11-24T06:29:12.681363+0000","last_undegraded":"2023-11-24T06:29:12.681363+0000","last_fullsized":"2023-11-24T06:29:12.681363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:07.278001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.291184+0000","last_change":"2023-11-24T06:29:11.285077+0000","last_active":"2023-11-24T06:29:12.291184+0000","last_peered":"2023-11-24T06:29:12.291184+0000","last_clean":"2023-11-24T06:29:12.291184+0000","last_became_active":"2023-11-24T06:29:11.284814+0000","last_became_peered":"2023-11-24T06:29:11.284814+0000","last_unstale":"2023-11-24T06:29:12.291184+0000","last_undegraded":"2023-11-24T06:29:12.291184+0000","last_fullsized":"2023-11-24T06:29:12.291184+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:48.087034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289601+0000","last_change":"2023-11-24T06:29:10.277251+0000","last_active":"2023-11-24T06:29:12.289601+0000","last_peered":"2023-11-24T06:29:12.289601+0000","last_clean":"2023-11-24T06:29:12.289601+0000","last_became_active":"2023-11-24T06:29:10.275293+0000","last_became_peered":"2023-11-24T06:29:10.275293+0000","last_unstale":"2023-11-24T06:29:12.289601+0000","last_undegraded":"2023-11-24T06:29:12.289601+0000","last_fullsized":"2023-11-24T06:29:12.289601+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:46.613802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681358+0000","last_change":"2023-11-24T06:29:11.292649+0000","last_active":"2023-11-24T06:29:12.681358+0000","last_peered":"2023-11-24T06:29:12.681358+0000","last_clean":"2023-11-24T06:29:12.681358+0000","last_became_active":"2023-11-24T06:29:11.292358+0000","last_became_peered":"2023-11-24T06:29:11.292358+0000","last_unstale":"2023-11-24T06:29:12.681358+0000","last_undegraded":"2023-11-24T06:29:12.681358+0000","last_fullsized":"2023-11-24T06:29:12.681358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:34.786099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289138+0000","last_change":"2023-11-24T06:29:10.271755+0000","last_active":"2023-11-24T06:29:12.289138+0000","last_peered":"2023-11-24T06:29:12.289138+0000","last_clean":"2023-11-24T06:29:12.289138+0000","last_became_active":"2023-11-24T06:29:10.271435+0000","last_became_peered":"2023-11-24T06:29:10.271435+0000","last_unstale":"2023-11-24T06:29:12.289138+0000","last_undegraded":"2023-11-24T06:29:12.289138+0000","last_fullsized":"2023-11-24T06:29:12.289138+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:51.485950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286034+0000","last_change":"2023-11-24T06:29:11.282118+0000","last_active":"2023-11-24T06:29:12.286034+0000","last_peered":"2023-11-24T06:29:12.286034+0000","last_clean":"2023-11-24T06:29:12.286034+0000","last_became_active":"2023-11-24T06:29:11.281574+0000","last_became_peered":"2023-11-24T06:29:11.281574+0000","last_unstale":"2023-11-24T06:29:12.286034+0000","last_undegraded":"2023-11-24T06:29:12.286034+0000","last_fullsized":"2023-11-24T06:29:12.286034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:00.733955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.289686+0000","last_change":"2023-11-24T06:29:10.284750+0000","last_active":"2023-11-24T06:29:12.289686+0000","last_peered":"2023-11-24T06:29:12.289686+0000","last_clean":"2023-11-24T06:29:12.289686+0000","last_became_active":"2023-11-24T06:29:10.284466+0000","last_became_peered":"2023-11-24T06:29:10.284466+0000","last_unstale":"2023-11-24T06:29:12.289686+0000","last_undegraded":"2023-11-24T06:29:12.289686+0000","last_fullsized":"2023-11-24T06:29:12.289686+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:37.847544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285546+0000","last_change":"2023-11-24T06:29:11.281785+0000","last_active":"2023-11-24T06:29:12.285546+0000","last_peered":"2023-11-24T06:29:12.285546+0000","last_clean":"2023-11-24T06:29:12.285546+0000","last_became_active":"2023-11-24T06:29:11.279634+0000","last_became_peered":"2023-11-24T06:29:11.279634+0000","last_unstale":"2023-11-24T06:29:12.285546+0000","last_undegraded":"2023-11-24T06:29:12.285546+0000","last_fullsized":"2023-11-24T06:29:12.285546+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:31.147590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682752+0000","last_change":"2023-11-24T06:29:10.274576+0000","last_active":"2023-11-24T06:29:12.682752+0000","last_peered":"2023-11-24T06:29:12.682752+0000","last_clean":"2023-11-24T06:29:12.682752+0000","last_became_active":"2023-11-24T06:29:10.274244+0000","last_became_peered":"2023-11-24T06:29:10.274244+0000","last_unstale":"2023-11-24T06:29:12.682752+0000","last_undegraded":"2023-11-24T06:29:12.682752+0000","last_fullsized":"2023-11-24T06:29:12.682752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:45.022343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.286210+0000","last_change":"2023-11-24T06:29:11.295929+0000","last_active":"2023-11-24T06:29:12.286210+0000","last_peered":"2023-11-24T06:29:12.286210+0000","last_clean":"2023-11-24T06:29:12.286210+0000","last_became_active":"2023-11-24T06:29:11.295404+0000","last_became_peered":"2023-11-24T06:29:11.295404+0000","last_unstale":"2023-11-24T06:29:12.286210+0000","last_undegraded":"2023-11-24T06:29:12.286210+0000","last_fullsized":"2023-11-24T06:29:12.286210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:46.036916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681399+0000","last_change":"2023-11-24T06:29:10.279578+0000","last_active":"2023-11-24T06:29:12.681399+0000","last_peered":"2023-11-24T06:29:12.681399+0000","last_clean":"2023-11-24T06:29:12.681399+0000","last_became_active":"2023-11-24T06:29:10.279224+0000","last_became_peered":"2023-11-24T06:29:10.279224+0000","last_unstale":"2023-11-24T06:29:12.681399+0000","last_undegraded":"2023-11-24T06:29:12.681399+0000","last_fullsized":"2023-11-24T06:29:12.681399+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:46.064061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285855+0000","last_change":"2023-11-24T06:29:11.291015+0000","last_active":"2023-11-24T06:29:12.285855+0000","last_peered":"2023-11-24T06:29:12.285855+0000","last_clean":"2023-11-24T06:29:12.285855+0000","last_became_active":"2023-11-24T06:29:11.290743+0000","last_became_peered":"2023-11-24T06:29:11.290743+0000","last_unstale":"2023-11-24T06:29:12.285855+0000","last_undegraded":"2023-11-24T06:29:12.285855+0000","last_fullsized":"2023-11-24T06:29:12.285855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:47.078633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288264+0000","last_change":"2023-11-24T06:29:10.280791+0000","last_active":"2023-11-24T06:29:12.288264+0000","last_peered":"2023-11-24T06:29:12.288264+0000","last_clean":"2023-11-24T06:29:12.288264+0000","last_became_active":"2023-11-24T06:29:10.280032+0000","last_became_peered":"2023-11-24T06:29:10.280032+0000","last_unstale":"2023-11-24T06:29:12.288264+0000","last_undegraded":"2023-11-24T06:29:12.288264+0000","last_fullsized":"2023-11-24T06:29:12.288264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:45.022343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327522+0000","last_change":"2023-11-24T06:29:11.287226+0000","last_active":"2023-11-24T06:29:12.327522+0000","last_peered":"2023-11-24T06:29:12.327522+0000","last_clean":"2023-11-24T06:29:12.327522+0000","last_became_active":"2023-11-24T06:29:11.284025+0000","last_became_peered":"2023-11-24T06:29:11.284025+0000","last_unstale":"2023-11-24T06:29:12.327522+0000","last_undegraded":"2023-11-24T06:29:12.327522+0000","last_fullsized":"2023-11-24T06:29:12.327522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:29.315454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288390+0000","last_change":"2023-11-24T06:29:10.283882+0000","last_active":"2023-11-24T06:29:12.288390+0000","last_peered":"2023-11-24T06:29:12.288390+0000","last_clean":"2023-11-24T06:29:12.288390+0000","last_became_active":"2023-11-24T06:29:10.283591+0000","last_became_peered":"2023-11-24T06:29:10.283591+0000","last_unstale":"2023-11-24T06:29:12.288390+0000","last_undegraded":"2023-11-24T06:29:12.288390+0000","last_fullsized":"2023-11-24T06:29:12.288390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:45.022343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.285761+0000","last_change":"2023-11-24T06:29:11.277568+0000","last_active":"2023-11-24T06:29:12.285761+0000","last_peered":"2023-11-24T06:29:12.285761+0000","last_clean":"2023-11-24T06:29:12.285761+0000","last_became_active":"2023-11-24T06:29:11.277383+0000","last_became_peered":"2023-11-24T06:29:11.277383+0000","last_unstale":"2023-11-24T06:29:12.285761+0000","last_undegraded":"2023-11-24T06:29:12.285761+0000","last_fullsized":"2023-11-24T06:29:12.285761+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:44.174589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681416+0000","last_change":"2023-11-24T06:29:10.279864+0000","last_active":"2023-11-24T06:29:12.681416+0000","last_peered":"2023-11-24T06:29:12.681416+0000","last_clean":"2023-11-24T06:29:12.681416+0000","last_became_active":"2023-11-24T06:29:10.279484+0000","last_became_peered":"2023-11-24T06:29:10.279484+0000","last_unstale":"2023-11-24T06:29:12.681416+0000","last_undegraded":"2023-11-24T06:29:12.681416+0000","last_fullsized":"2023-11-24T06:29:12.681416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:33:05.543857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.291358+0000","last_change":"2023-11-24T06:29:11.293523+0000","last_active":"2023-11-24T06:29:12.291358+0000","last_peered":"2023-11-24T06:29:12.291358+0000","last_clean":"2023-11-24T06:29:12.291358+0000","last_became_active":"2023-11-24T06:29:11.293244+0000","last_became_peered":"2023-11-24T06:29:11.293244+0000","last_unstale":"2023-11-24T06:29:12.291358+0000","last_undegraded":"2023-11-24T06:29:12.291358+0000","last_fullsized":"2023-11-24T06:29:12.291358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:06.364499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682766+0000","last_change":"2023-11-24T06:29:10.277004+0000","last_active":"2023-11-24T06:29:12.682766+0000","last_peered":"2023-11-24T06:29:12.682766+0000","last_clean":"2023-11-24T06:29:12.682766+0000","last_became_active":"2023-11-24T06:29:10.276327+0000","last_became_peered":"2023-11-24T06:29:10.276327+0000","last_unstale":"2023-11-24T06:29:12.682766+0000","last_undegraded":"2023-11-24T06:29:12.682766+0000","last_fullsized":"2023-11-24T06:29:12.682766+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:46.064061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682524+0000","last_change":"2023-11-24T06:29:11.301942+0000","last_active":"2023-11-24T06:29:12.682524+0000","last_peered":"2023-11-24T06:29:12.682524+0000","last_clean":"2023-11-24T06:29:12.682524+0000","last_became_active":"2023-11-24T06:29:11.301716+0000","last_became_peered":"2023-11-24T06:29:11.301716+0000","last_unstale":"2023-11-24T06:29:12.682524+0000","last_undegraded":"2023-11-24T06:29:12.682524+0000","last_fullsized":"2023-11-24T06:29:12.682524+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:52.500523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681398+0000","last_change":"2023-11-24T06:29:10.267944+0000","last_active":"2023-11-24T06:29:12.681398+0000","last_peered":"2023-11-24T06:29:12.681398+0000","last_clean":"2023-11-24T06:29:12.681398+0000","last_became_active":"2023-11-24T06:29:10.267632+0000","last_became_peered":"2023-11-24T06:29:10.267632+0000","last_unstale":"2023-11-24T06:29:12.681398+0000","last_undegraded":"2023-11-24T06:29:12.681398+0000","last_fullsized":"2023-11-24T06:29:12.681398+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:42.651516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681315+0000","last_change":"2023-11-24T06:29:11.290691+0000","last_active":"2023-11-24T06:29:12.681315+0000","last_peered":"2023-11-24T06:29:12.681315+0000","last_clean":"2023-11-24T06:29:12.681315+0000","last_became_active":"2023-11-24T06:29:11.290400+0000","last_became_peered":"2023-11-24T06:29:11.290400+0000","last_unstale":"2023-11-24T06:29:12.681315+0000","last_undegraded":"2023-11-24T06:29:12.681315+0000","last_fullsized":"2023-11-24T06:29:12.681315+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:46.036916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682990+0000","last_change":"2023-11-24T06:29:10.272285+0000","last_active":"2023-11-24T06:29:12.682990+0000","last_peered":"2023-11-24T06:29:12.682990+0000","last_clean":"2023-11-24T06:29:12.682990+0000","last_became_active":"2023-11-24T06:29:10.271851+0000","last_became_peered":"2023-11-24T06:29:10.271851+0000","last_unstale":"2023-11-24T06:29:12.682990+0000","last_undegraded":"2023-11-24T06:29:12.682990+0000","last_fullsized":"2023-11-24T06:29:12.682990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:37.847544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.682681+0000","last_change":"2023-11-24T06:29:11.293391+0000","last_active":"2023-11-24T06:29:12.682681+0000","last_peered":"2023-11-24T06:29:12.682681+0000","last_clean":"2023-11-24T06:29:12.682681+0000","last_became_active":"2023-11-24T06:29:11.292332+0000","last_became_peered":"2023-11-24T06:29:11.292332+0000","last_unstale":"2023-11-24T06:29:12.682681+0000","last_undegraded":"2023-11-24T06:29:12.682681+0000","last_fullsized":"2023-11-24T06:29:12.682681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:48.241504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680674+0000","last_change":"2023-11-24T06:29:10.279399+0000","last_active":"2023-11-24T06:29:12.680674+0000","last_peered":"2023-11-24T06:29:12.680674+0000","last_clean":"2023-11-24T06:29:12.680674+0000","last_became_active":"2023-11-24T06:29:10.278981+0000","last_became_peered":"2023-11-24T06:29:10.278981+0000","last_unstale":"2023-11-24T06:29:12.680674+0000","last_undegraded":"2023-11-24T06:29:12.680674+0000","last_fullsized":"2023-11-24T06:29:12.680674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:04.116234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680305+0000","last_change":"2023-11-24T06:29:11.310443+0000","last_active":"2023-11-24T06:29:12.680305+0000","last_peered":"2023-11-24T06:29:12.680305+0000","last_clean":"2023-11-24T06:29:12.680305+0000","last_became_active":"2023-11-24T06:29:11.310183+0000","last_became_peered":"2023-11-24T06:29:11.310183+0000","last_unstale":"2023-11-24T06:29:12.680305+0000","last_undegraded":"2023-11-24T06:29:12.680305+0000","last_fullsized":"2023-11-24T06:29:12.680305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:02.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288508+0000","last_change":"2023-11-24T06:29:10.270585+0000","last_active":"2023-11-24T06:29:12.288508+0000","last_peered":"2023-11-24T06:29:12.288508+0000","last_clean":"2023-11-24T06:29:12.288508+0000","last_became_active":"2023-11-24T06:29:10.270168+0000","last_became_peered":"2023-11-24T06:29:10.270168+0000","last_unstale":"2023-11-24T06:29:12.288508+0000","last_undegraded":"2023-11-24T06:29:12.288508+0000","last_fullsized":"2023-11-24T06:29:12.288508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:07:22.512103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.288327+0000","last_change":"2023-11-24T06:29:11.306265+0000","last_active":"2023-11-24T06:29:12.288327+0000","last_peered":"2023-11-24T06:29:12.288327+0000","last_clean":"2023-11-24T06:29:12.288327+0000","last_became_active":"2023-11-24T06:29:11.305921+0000","last_became_peered":"2023-11-24T06:29:11.305921+0000","last_unstale":"2023-11-24T06:29:12.288327+0000","last_undegraded":"2023-11-24T06:29:12.288327+0000","last_fullsized":"2023-11-24T06:29:12.288327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:21.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.680549+0000","last_change":"2023-11-24T06:29:10.282578+0000","last_active":"2023-11-24T06:29:12.680549+0000","last_peered":"2023-11-24T06:29:12.680549+0000","last_clean":"2023-11-24T06:29:12.680549+0000","last_became_active":"2023-11-24T06:29:10.282324+0000","last_became_peered":"2023-11-24T06:29:10.282324+0000","last_unstale":"2023-11-24T06:29:12.680549+0000","last_undegraded":"2023-11-24T06:29:12.680549+0000","last_fullsized":"2023-11-24T06:29:12.680549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:33.771527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.327982+0000","last_change":"2023-11-24T06:29:11.280873+0000","last_active":"2023-11-24T06:29:12.327982+0000","last_peered":"2023-11-24T06:29:12.327982+0000","last_clean":"2023-11-24T06:29:12.327982+0000","last_became_active":"2023-11-24T06:29:11.279668+0000","last_became_peered":"2023-11-24T06:29:11.279668+0000","last_unstale":"2023-11-24T06:29:12.327982+0000","last_undegraded":"2023-11-24T06:29:12.327982+0000","last_fullsized":"2023-11-24T06:29:12.327982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:10.229254+0000","last_clean_scrub_stamp":"2023-11-24T06:29:10.229254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:24.991791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T06:29:12.681323+0000","last_change":"2023-11-24T06:29:10.276596+0000","last_active":"2023-11-24T06:29:12.681323+0000","last_peered":"2023-11-24T06:29:12.681323+0000","last_clean":"2023-11-24T06:29:12.681323+0000","last_became_active":"2023-11-24T06:29:10.276278+0000","last_became_peered":"2023-11-24T06:29:10.276278+0000","last_unstale":"2023-11-24T06:29:12.681323+0000","last_undegraded":"2023-11-24T06:29:12.681323+0000","last_fullsized":"2023-11-24T06:29:12.681323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:09.214682+0000","last_clean_scrub_stamp":"2023-11-24T06:29:09.214682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:09.198463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:29:11.261762+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:29:11.261762+0000","last_peered":"2023-11-24T06:29:11.261762+0000","last_clean":"2023-11-24T06:29:11.261762+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:29:11.261762+0000","last_undegraded":"2023-11-24T06:29:11.261762+0000","last_fullsized":"2023-11-24T06:29:11.261762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":750119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.274}]},{"osd":2,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.78},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.228}]},{"osd":3,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.335}]},{"osd":4,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.921}]},{"osd":5,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":2.093},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.853}]},{"osd":6,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":2.032},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":2.0129999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":160277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.526},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.6919999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.601}]},{"osd":3,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.288}]},{"osd":4,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.397},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.3420000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.6359999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.427},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.111}]}]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":160333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.2450000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.5449999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.175}]},{"osd":4,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.9119999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.3600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.0299999999999998,"5min":3.0299999999999998,"15min":3.0299999999999998},"last":1.7370000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.546,"15min":1.546},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.5840000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":159765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.6000000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.097},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.95999999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.353},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.671}]},{"osd":3,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.633}]},{"osd":5,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":4.3979999999999997,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":4.2060000000000004,"5min":4.2060000000000004,"15min":4.2060000000000004},"last":1.706}]},{"osd":6,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.0339999999999998,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.335,"5min":4.335,"15min":4.335},"last":1.9630000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":3.964,"5min":3.964,"15min":3.964},"last":2.016},{"interface":"front","average":{"1min":1.661,"5min":1.661,"15min":1.661},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.8809999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":1.9970000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.0669999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.214},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.389},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.298}]},{"osd":5,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.538},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.6319999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.502},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.663}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":159731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.83899999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.1779999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.996}]},{"osd":4,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.51},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.3839999999999999,"5min":3.3839999999999999,"15min":3.3839999999999999},"last":1.4450000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":1.4059999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.218},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.375}]},{"osd":7,"last update":"Fri Nov 24 06:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.9969999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":1.3540000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":161295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.032}]},{"osd":2,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.6870000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":1.615}]},{"osd":5,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.163},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.454}]},{"osd":6,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.52},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.5660000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:36 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.3999999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":750029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.119},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.02}]},{"osd":2,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.179},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.504}]},{"osd":5,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.333},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1}]},{"osd":6,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":3.3730000000000002,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.4039999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.3700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:16.261 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:29:16.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:29:16.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.538+0000 7f17ab101640 1 Processor -- start 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.538+0000 7f17ab101640 1 -- start start 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.539+0000 7f17ab101640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a4160c80 0x7f17a40b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.539+0000 7f17ab101640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f17a40cdd40 0x7f17a40cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.539+0000 7f17ab101640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a40d21d0 con 0x7f17a40d00d0 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.539+0000 7f17ab101640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a4161060 con 0x7f17a40cdd40 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.539+0000 7f17ab101640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40cf2c0 con 0x7f17a4160c80 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.540+0000 7f17a969e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f17a40d00d0 0x7f17a40d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59526/0 (socket says 172.21.15.64:59526) 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.540+0000 7f17a969e640 1 -- 172.21.15.64:0/1626988477 learned_addr learned my addr 172.21.15.64:0/1626988477 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.540+0000 7f17a8e9d640 1 --2- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a4160c80 0x7f17a40b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.540+0000 7f179bfff640 1 --2- 172.21.15.64:0/1626988477 >> v2:172.21.15.162:3300/0 conn(0x7f17a40cdd40 0x7f17a40cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f179b7fe640 1 -- 172.21.15.64:0/1626988477 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136444150 0 0) 0x7f179c001110 con 0x7f17a40d00d0 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f179b7fe640 1 -- 172.21.15.64:0/1626988477 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f178c003520 con 0x7f17a40d00d0 2023-11-24T06:29:16.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f179b7fe640 1 -- 172.21.15.64:0/1626988477 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2581232319 0 0) 0x7f178c003520 con 0x7f17a40d00d0 2023-11-24T06:29:16.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f179bfff640 1 -- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a4160c80 msgr2=0x7f17a40b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:16.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f179bfff640 1 --2- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a4160c80 0x7f17a40b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f179bfff640 1 -- 172.21.15.64:0/1626988477 >> v1:172.21.15.64:6789/0 conn(0x7f17a40d00d0 legacy=0x7f17a40d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f179bfff640 1 -- 172.21.15.64:0/1626988477 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a40cee30 con 0x7f17a40cdd40 2023-11-24T06:29:16.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.541+0000 7f17a8e9d640 1 --2- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a4160c80 0x7f17a40b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:16.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.542+0000 7f179bfff640 1 --2- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40cdd40 0x7f17a40cfb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17940097a0 tx=0x7f179402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e890dc29da8ca12f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.543+0000 7f179b7fe640 1 -- 172.21.15.64:0/1626988477 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c001800 con 0x7f17a40cdd40 2023-11-24T06:29:16.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.543+0000 7f179b7fe640 1 -- 172.21.15.64:0/1626988477 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179402e050 con 0x7f17a40cdd40 2023-11-24T06:29:16.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.543+0000 7f179b7fe640 1 -- 172.21.15.64:0/1626988477 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794030730 con 0x7f17a40cdd40 2023-11-24T06:29:16.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.543+0000 7f17ab101640 1 -- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40cdd40 msgr2=0x7f17a40cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.543+0000 7f17ab101640 1 --2- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40cdd40 0x7f17a40cfb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17940097a0 tx=0x7f179402b790 comp rx=0 tx=0).stop 2023-11-24T06:29:16.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.544+0000 7f17ab101640 1 -- 172.21.15.64:0/1626988477 shutdown_connections 2023-11-24T06:29:16.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.544+0000 7f17ab101640 1 --2- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40cdd40 0x7f17a40cfb90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.544+0000 7f17ab101640 1 --2- 172.21.15.64:0/1626988477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a4160c80 0x7f17a40b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.544+0000 7f17ab101640 1 -- 172.21.15.64:0/1626988477 >> 172.21.15.64:0/1626988477 conn(0x7f17a409c590 msgr2=0x7f17a40d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:16.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.544+0000 7f17ab101640 1 -- 172.21.15.64:0/1626988477 shutdown_connections 2023-11-24T06:29:16.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.544+0000 7f17ab101640 1 -- 172.21.15.64:0/1626988477 wait complete. 2023-11-24T06:29:16.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.545+0000 7f17ab101640 1 Processor -- start 2023-11-24T06:29:16.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.546+0000 7f17ab101640 1 -- start start 2023-11-24T06:29:16.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.546+0000 7f17ab101640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a40cdd40 0x7f17a4153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.546+0000 7f17ab101640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40d00d0 0x7f17a40cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.546+0000 7f17ab101640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a4160c80 0x7f17a40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.546+0000 7f17ab101640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40cc540 con 0x7f17a4160c80 2023-11-24T06:29:16.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.546+0000 7f17ab101640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40cc260 con 0x7f17a40d00d0 2023-11-24T06:29:16.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.546+0000 7f17ab101640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a41f6f10 con 0x7f17a40cdd40 2023-11-24T06:29:16.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.547+0000 7f17a969e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a4160c80 0x7f17a40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.547+0000 7f17a8e9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a40cdd40 0x7f17a4153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.547+0000 7f179bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40d00d0 0x7f17a40cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.547+0000 7f17a969e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a4160c80 0x7f17a40d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37366/0 (socket says 172.21.15.64:37366) 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.547+0000 7f17a8e9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a40cdd40 0x7f17a4153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:43642/0 (socket says 172.21.15.64:43642) 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.547+0000 7f17a969e640 1 -- 172.21.15.64:0/127845859 learned_addr learned my addr 172.21.15.64:0/127845859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17a969e640 1 -- 172.21.15.64:0/127845859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a40cdd40 msgr2=0x7f17a4153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17a969e640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a40cdd40 0x7f17a4153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17a969e640 1 -- 172.21.15.64:0/127845859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40d00d0 msgr2=0x7f17a40cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17a969e640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40d00d0 0x7f17a40cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17a969e640 1 -- 172.21.15.64:0/127845859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0009a70 con 0x7f17a4160c80 2023-11-24T06:29:16.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17a969e640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a4160c80 0x7f17a40d34d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c750 tx=0x7f179c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17997fa640 1 -- 172.21.15.64:0/127845859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c005ad0 con 0x7f17a4160c80 2023-11-24T06:29:16.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17997fa640 1 -- 172.21.15.64:0/127845859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c005c50 con 0x7f17a4160c80 2023-11-24T06:29:16.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17997fa640 1 -- 172.21.15.64:0/127845859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c007070 con 0x7f17a4160c80 2023-11-24T06:29:16.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.548+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1794009400 con 0x7f17a4160c80 2023-11-24T06:29:16.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.549+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a41f74d0 con 0x7f17a4160c80 2023-11-24T06:29:16.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.550+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a41f76f0 con 0x7f17a4160c80 2023-11-24T06:29:16.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.551+0000 7f17997fa640 1 -- 172.21.15.64:0/127845859 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f179c0071d0 con 0x7f17a4160c80 2023-11-24T06:29:16.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.552+0000 7f17997fa640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f179006ae40 0x7f179006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.552+0000 7f17997fa640 1 -- 172.21.15.64:0/127845859 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f179c088fe0 con 0x7f17a4160c80 2023-11-24T06:29:16.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.552+0000 7f17a8e9d640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f179006ae40 0x7f179006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.552+0000 7f17a8e9d640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f179006ae40 0x7f179006d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009fd0 tx=0x7f17a000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.556+0000 7f17997fa640 1 -- 172.21.15.64:0/127845859 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f179c054190 con 0x7f17a4160c80 2023-11-24T06:29:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.821+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f17a40d21d0 con 0x7f17a4160c80 2023-11-24T06:29:16.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.826+0000 7f17997fa640 1 -- 172.21.15.64:0/127845859 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f17a40d21d0 con 0x7f17a4160c80 2023-11-24T06:29:16.829 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T06:29:16.829 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAWQmBlHzE7KRAAcsll2ZSOmVWCava8K2aOxA== 2023-11-24T06:29:16.829 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2023-11-24T06:29:16.829 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2023-11-24T06:29:16.829 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:29:16.830 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-11-24T06:29:16.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.829+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f179006ae40 msgr2=0x7f179006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.829+0000 7f17ab101640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f179006ae40 0x7f179006d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009fd0 tx=0x7f17a000b040 comp rx=0 tx=0).stop 2023-11-24T06:29:16.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.829+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a4160c80 msgr2=0x7f17a40d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.829+0000 7f17ab101640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a4160c80 0x7f17a40d34d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c750 tx=0x7f179c00cc10 comp rx=0 tx=0).stop 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 shutdown_connections 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f179006ae40 0x7f179006d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a4160c80 0x7f17a40d34d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a40d00d0 0x7f17a40cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 --2- 172.21.15.64:0/127845859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17a40cdd40 0x7f17a4153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 >> 172.21.15.64:0/127845859 conn(0x7f17a409c590 msgr2=0x7f17a406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 shutdown_connections 2023-11-24T06:29:16.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:16.830+0000 7f17ab101640 1 -- 172.21.15.64:0/127845859 wait complete. 2023-11-24T06:29:16.851 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:17.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.074+0000 7fbec96d1640 1 Processor -- start 2023-11-24T06:29:17.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.074+0000 7fbec96d1640 1 -- start start 2023-11-24T06:29:17.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec96d1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbec40cc500 0x7fbec40cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec96d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbec406d160 0x7fbec406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec96d1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec4070e50 con 0x7fbec4143c40 2023-11-24T06:29:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec96d1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec40723c0 con 0x7fbec406d160 2023-11-24T06:29:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec96d1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec40cced0 con 0x7fbec40cc500 2023-11-24T06:29:17.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec8ed0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbec406d160 0x7fbec406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec8ed0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbec406d160 0x7fbec406d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37376/0 (socket says 172.21.15.64:37376) 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.076+0000 7fbec8ed0640 1 -- 172.21.15.64:0/3018338338 learned_addr learned my addr 172.21.15.64:0/3018338338 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.077+0000 7fbebb7fe640 1 --2- 172.21.15.64:0/3018338338 >> v2:172.21.15.162:3300/0 conn(0x7fbec40cc500 0x7fbec40cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.077+0000 7fbec8ed0640 1 -- 172.21.15.64:0/3018338338 >> v2:172.21.15.162:3300/0 conn(0x7fbec40cc500 msgr2=0x7fbec40cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.077+0000 7fbec8ed0640 1 --2- 172.21.15.64:0/3018338338 >> v2:172.21.15.162:3300/0 conn(0x7fbec40cc500 0x7fbec40cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.077+0000 7fbec8ed0640 1 -- 172.21.15.64:0/3018338338 >> v1:172.21.15.162:6789/0 conn(0x7fbec4143c40 legacy=0x7fbec4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.077+0000 7fbec8ed0640 1 -- 172.21.15.64:0/3018338338 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec4144020 con 0x7fbec406d160 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.078+0000 7fbec8ed0640 1 --2- 172.21.15.64:0/3018338338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec406d560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbec40b44a0 tx=0x7fbebc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b26981d39008589 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.078+0000 7fbec37fe640 1 -- 172.21.15.64:0/3018338338 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc0317e0 con 0x7fbec406d160 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.078+0000 7fbec37fe640 1 -- 172.21.15.64:0/3018338338 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc031940 con 0x7fbec406d160 2023-11-24T06:29:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.078+0000 7fbec37fe640 1 -- 172.21.15.64:0/3018338338 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc031cd0 con 0x7fbec406d160 2023-11-24T06:29:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 -- 172.21.15.64:0/3018338338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 msgr2=0x7fbec406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 --2- 172.21.15.64:0/3018338338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec406d560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbec40b44a0 tx=0x7fbebc02b880 comp rx=0 tx=0).stop 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 -- 172.21.15.64:0/3018338338 shutdown_connections 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 --2- 172.21.15.64:0/3018338338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec406d560 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 --2- 172.21.15.64:0/3018338338 >> v2:172.21.15.162:3300/0 conn(0x7fbec40cc500 0x7fbec40cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 -- 172.21.15.64:0/3018338338 >> 172.21.15.64:0/3018338338 conn(0x7fbec406a800 msgr2=0x7fbec406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 -- 172.21.15.64:0/3018338338 shutdown_connections 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.079+0000 7fbec96d1640 1 -- 172.21.15.64:0/3018338338 wait complete. 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.081+0000 7fbec96d1640 1 Processor -- start 2023-11-24T06:29:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.082+0000 7fbec96d1640 1 -- start start 2023-11-24T06:29:17.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec96d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec41f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec96d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbec40cc500 0x7fbec40af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec96d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec4143c40 0x7fbec40af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec96d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4156ee0 con 0x7fbec406d160 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec96d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41f7260 con 0x7fbec4143c40 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec96d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4153150 con 0x7fbec40cc500 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec41f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec41f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37392/0 (socket says 172.21.15.64:37392) 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.083+0000 7fbec3fff640 1 -- 172.21.15.64:0/1765355862 learned_addr learned my addr 172.21.15.64:0/1765355862 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec3fff640 1 -- 172.21.15.64:0/1765355862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbec40cc500 msgr2=0x7fbec40af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec3fff640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbec40cc500 0x7fbec40af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec3fff640 1 -- 172.21.15.64:0/1765355862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec4143c40 msgr2=0x7fbec40af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec3fff640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec4143c40 0x7fbec40af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec3fff640 1 -- 172.21.15.64:0/1765355862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc0094d0 con 0x7fbec406d160 2023-11-24T06:29:17.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec3fff640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec41f2500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400b420 tx=0x7fbeb400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:17.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec17fa640 1 -- 172.21.15.64:0/1765355862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4011830 con 0x7fbec406d160 2023-11-24T06:29:17.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec17fa640 1 -- 172.21.15.64:0/1765355862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb4011990 con 0x7fbec406d160 2023-11-24T06:29:17.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec17fa640 1 -- 172.21.15.64:0/1765355862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4011d40 con 0x7fbec406d160 2023-11-24T06:29:17.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec4156960 con 0x7fbec406d160 2023-11-24T06:29:17.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.084+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec4152b00 con 0x7fbec406d160 2023-11-24T06:29:17.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.085+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec4067450 con 0x7fbec406d160 2023-11-24T06:29:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.086+0000 7fbec17fa640 1 -- 172.21.15.64:0/1765355862 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbeb401b460 con 0x7fbec406d160 2023-11-24T06:29:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.087+0000 7fbec17fa640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbea406ae40 0x7fbea406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.087+0000 7fbec17fa640 1 -- 172.21.15.64:0/1765355862 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeb4089340 con 0x7fbec406d160 2023-11-24T06:29:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.087+0000 7fbebb7fe640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbea406ae40 0x7fbea406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:17.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.088+0000 7fbebb7fe640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbea406ae40 0x7fbea406d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0070b0 tx=0x7fbeac007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:17.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.092+0000 7fbec17fa640 1 -- 172.21.15.64:0/1765355862 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbeb4054420 con 0x7fbec406d160 2023-11-24T06:29:17.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.325+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbec40cd930 con 0x7fbec406d160 2023-11-24T06:29:17.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.327+0000 7fbec17fa640 1 -- 172.21.15.64:0/1765355862 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fbeb4057a40 con 0x7fbec406d160 2023-11-24T06:29:17.330 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:17.330 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3831306721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3831306721},{"type":"v1","addr":"172.21.15.162:6834","nonce":3831306721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1401622125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1401622125},{"type":"v1","addr":"172.21.15.162:6835","nonce":1401622125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2378368513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6835","nonce":2378368513},{"type":"v1","addr":"172.21.15.64:6837","nonce":2378368513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:10.724367+0000","modified":"2023-11-24T06:29:13.855414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6836/2541249514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2541249514},{"type":"v1","addr":"172.21.15.64:6836","nonce":2541249514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:29:17.330 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2023-11-24T06:29:17.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.329+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbea406ae40 msgr2=0x7fbea406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.329+0000 7fbec96d1640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbea406ae40 0x7fbea406d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0070b0 tx=0x7fbeac007040 comp rx=0 tx=0).stop 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.329+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 msgr2=0x7fbec41f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.329+0000 7fbec96d1640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec41f2500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400b420 tx=0x7fbeb400b8e0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 shutdown_connections 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbea406ae40 0x7fbea406d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec4143c40 0x7fbec40af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbec40cc500 0x7fbec40af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 --2- 172.21.15.64:0/1765355862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbec406d160 0x7fbec41f2500 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 >> 172.21.15.64:0/1765355862 conn(0x7fbec406a800 msgr2=0x7fbec40aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 shutdown_connections 2023-11-24T06:29:17.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.330+0000 7fbec96d1640 1 -- 172.21.15.64:0/1765355862 wait complete. 2023-11-24T06:29:17.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:29:10.724367+0000', 'modified': '2023-11-24T06:29:13.855414+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5199}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5199': {'gid': 5199, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.64:6836/2541249514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 2541249514}, {'type': 'v1', 'addr': '172.21.15.64:6836', 'nonce': 2541249514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:29:17.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:29:17.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T06:29:17.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.986+0000 7f8355cea640 1 Processor -- start 2023-11-24T06:29:17.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.987+0000 7f8355cea640 1 -- start start 2023-11-24T06:29:17.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.988+0000 7f8355cea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8350162430 0x7f83500b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.988+0000 7f8355cea640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8350162810 0x7f8350071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8355cea640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83500702e0 con 0x7f83500b3650 2023-11-24T06:29:17.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8355cea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83500705c0 con 0x7f8350162430 2023-11-24T06:29:17.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8355cea640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83500708a0 con 0x7f8350162810 2023-11-24T06:29:17.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8354ce8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8350162430 0x7f83500b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:17.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8354ce8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8350162430 0x7f83500b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37406/0 (socket says 172.21.15.64:37406) 2023-11-24T06:29:17.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8354ce8640 1 -- 172.21.15.64:0/2293230686 learned_addr learned my addr 172.21.15.64:0/2293230686 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:17.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8354ce8640 1 -- 172.21.15.64:0/2293230686 >> v2:172.21.15.162:3300/0 conn(0x7f8350162810 msgr2=0x7f8350071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:17.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8354ce8640 1 --2- 172.21.15.64:0/2293230686 >> v2:172.21.15.162:3300/0 conn(0x7f8350162810 0x7f8350071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8354ce8640 1 -- 172.21.15.64:0/2293230686 >> v1:172.21.15.162:6789/0 conn(0x7f83500b3650 legacy=0x7f83501650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:17.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.989+0000 7f8354ce8640 1 -- 172.21.15.64:0/2293230686 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350070b80 con 0x7f8350162430 2023-11-24T06:29:17.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.991+0000 7f8354ce8640 1 --2- 172.21.15.64:0/2293230686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350162430 0x7f83500b3110 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f835015c090 tx=0x7f834002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ba4320e233b0b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:17.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.991+0000 7f834f7fe640 1 -- 172.21.15.64:0/2293230686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834002ca80 con 0x7f8350162430 2023-11-24T06:29:17.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.991+0000 7f834f7fe640 1 -- 172.21.15.64:0/2293230686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834002cbe0 con 0x7f8350162430 2023-11-24T06:29:17.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.991+0000 7f834f7fe640 1 -- 172.21.15.64:0/2293230686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400343f0 con 0x7f8350162430 2023-11-24T06:29:17.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 -- 172.21.15.64:0/2293230686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350162430 msgr2=0x7f83500b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 --2- 172.21.15.64:0/2293230686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350162430 0x7f83500b3110 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f835015c090 tx=0x7f834002bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 -- 172.21.15.64:0/2293230686 shutdown_connections 2023-11-24T06:29:17.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 --2- 172.21.15.64:0/2293230686 >> v2:172.21.15.162:3300/0 conn(0x7f8350162810 0x7f8350071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 --2- 172.21.15.64:0/2293230686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350162430 0x7f83500b3110 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 -- 172.21.15.64:0/2293230686 >> 172.21.15.64:0/2293230686 conn(0x7f835006a800 msgr2=0x7f835006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:17.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 -- 172.21.15.64:0/2293230686 shutdown_connections 2023-11-24T06:29:17.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.992+0000 7f8355cea640 1 -- 172.21.15.64:0/2293230686 wait complete. 2023-11-24T06:29:17.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.994+0000 7f8355cea640 1 Processor -- start 2023-11-24T06:29:17.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.994+0000 7f8355cea640 1 -- start start 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f8355cea640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83500b3650 0x7f835015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f8355cea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8350162430 0x7f8350159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f8355cea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350165bb0 0x7f835015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f8355cea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8350159b10 con 0x7f8350165bb0 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f8355cea640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83501ec0d0 con 0x7f83500b3650 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f8355cea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83501ec3b0 con 0x7f8350162430 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f83554e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350165bb0 0x7f835015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f83554e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350165bb0 0x7f835015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37416/0 (socket says 172.21.15.64:37416) 2023-11-24T06:29:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.995+0000 7f83554e9640 1 -- 172.21.15.64:0/2256808887 learned_addr learned my addr 172.21.15.64:0/2256808887 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f83554e9640 1 -- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8350162430 msgr2=0x7f8350159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f834ffff640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8350162430 0x7f8350159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f8354ce8640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83500b3650 0x7f835015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f83554e9640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8350162430 0x7f8350159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f83554e9640 1 -- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83500b3650 msgr2=0x7f835015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f83554e9640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83500b3650 0x7f835015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f83554e9640 1 -- 172.21.15.64:0/2256808887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340009b30 con 0x7f8350165bb0 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f834ffff640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8350162430 0x7f8350159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f8354ce8640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83500b3650 0x7f835015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.996+0000 7f83554e9640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350165bb0 0x7f835015b2b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f834800c360 tx=0x7f834800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.997+0000 7f834d7fa640 1 -- 172.21.15.64:0/2256808887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480035f0 con 0x7f8350165bb0 2023-11-24T06:29:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.997+0000 7f834d7fa640 1 -- 172.21.15.64:0/2256808887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348003750 con 0x7f8350165bb0 2023-11-24T06:29:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.997+0000 7f834d7fa640 1 -- 172.21.15.64:0/2256808887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348003b40 con 0x7f8350165bb0 2023-11-24T06:29:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.997+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83501ec690 con 0x7f8350165bb0 2023-11-24T06:29:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.997+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835015c640 con 0x7f8350165bb0 2023-11-24T06:29:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:17.999+0000 7f834d7fa640 1 -- 172.21.15.64:0/2256808887 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8348003ca0 con 0x7f8350165bb0 2023-11-24T06:29:18.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.000+0000 7f834d7fa640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f833c06ae40 0x7f833c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.000+0000 7f834d7fa640 1 -- 172.21.15.64:0/2256808887 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8348088880 con 0x7f8350165bb0 2023-11-24T06:29:18.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.000+0000 7f8354ce8640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f833c06ae40 0x7f833c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.001+0000 7f8359bc3740 1 Processor -- start 2023-11-24T06:29:18.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.001+0000 7f8359bc3740 1 -- start start 2023-11-24T06:29:18.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.001+0000 7f8354ce8640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f833c06ae40 0x7f833c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8340035000 tx=0x7f8340030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe4f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 0x5636dfe4e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4e7b0 0x5636dfe92730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f8359bc3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfdad880 con 0x5636dfe4e7b0 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfdbcae0 con 0x5636dfe4f070 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636dfdba1d0 con 0x5636dfe49d70 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f83554e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4e7b0 0x5636dfe92730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f83554e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4e7b0 0x5636dfe92730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37430/0 (socket says 172.21.15.64:37430) 2023-11-24T06:29:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.002+0000 7f83554e9640 1 -- 172.21.15.64:0/1969451490 learned_addr learned my addr 172.21.15.64:0/1969451490 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f834ffff640 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 0x5636dfe4e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f83554e9640 1 -- 172.21.15.64:0/1969451490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 msgr2=0x5636dfe4e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f83554e9640 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 0x5636dfe4e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f83554e9640 1 -- 172.21.15.64:0/1969451490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 msgr2=0x5636dfe4f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f83554e9640 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe4f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f83554e9640 1 -- 172.21.15.64:0/1969451490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636dfd7d860 con 0x5636dfe4e7b0 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f83554e9640 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4e7b0 0x5636dfe92730 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f83480050a0 tx=0x7f8348057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ca453d9be7f6a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f8336ffd640 1 -- 172.21.15.64:0/1969451490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834805f8e0 con 0x5636dfe4e7b0 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f8336ffd640 1 -- 172.21.15.64:0/1969451490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348055980 con 0x5636dfe4e7b0 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.003+0000 7f8336ffd640 1 -- 172.21.15.64:0/1969451490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834805f450 con 0x5636dfe4e7b0 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 -- 172.21.15.64:0/1969451490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4e7b0 msgr2=0x5636dfe92730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4e7b0 0x5636dfe92730 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f83480050a0 tx=0x7f8348057040 comp rx=0 tx=0).stop 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 -- 172.21.15.64:0/1969451490 shutdown_connections 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4e7b0 0x5636dfe92730 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 0x5636dfe4e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 --2- 172.21.15.64:0/1969451490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe4f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 -- 172.21.15.64:0/1969451490 >> 172.21.15.64:0/1969451490 conn(0x5636dfe10c70 msgr2=0x5636dfe94b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 -- 172.21.15.64:0/1969451490 shutdown_connections 2023-11-24T06:29:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.004+0000 7f8359bc3740 1 -- 172.21.15.64:0/1969451490 wait complete. 2023-11-24T06:29:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.005+0000 7f8359bc3740 1 -- start start 2023-11-24T06:29:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.005+0000 7f8359bc3740 1 Processor -- start 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe49d70 0x5636dfe5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe9c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe62380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8359bc3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfdb5ac0 con 0x5636dfe49d70 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfd522f0 con 0x5636dfe4f070 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636dfd6fc70 con 0x5636dfe4e7b0 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8359bc3740 10 client.0 _ll_register_callbacks cb 0x5636dfd743b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8354ce8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe49d70 0x5636dfe5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8354ce8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe49d70 0x5636dfe5fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37436/0 (socket says 172.21.15.64:37436) 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.006+0000 7f8354ce8640 1 -- 172.21.15.64:0/295767804 learned_addr learned my addr 172.21.15.64:0/295767804 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f83554e9640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe62380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f834ffff640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe9c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f8354ce8640 1 -- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 msgr2=0x5636dfe9c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f8354ce8640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe9c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f8354ce8640 1 -- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 msgr2=0x5636dfe62380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f8354ce8640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe62380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f8354ce8640 1 -- 172.21.15.64:0/295767804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835015c640 con 0x5636dfe49d70 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f83554e9640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe62380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f8354ce8640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe49d70 0x5636dfe5fcc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8340036040 tx=0x7f8340004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508d5a8d9e33eea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f832f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f832f7fe640 1 -- 172.21.15.64:0/295767804 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340046070 con 0x5636dfe49d70 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636dfdaeaa0 con 0x5636dfe49d70 2023-11-24T06:29:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.007+0000 7f832f7fe640 1 -- 172.21.15.64:0/295767804 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f834003e870 con 0x5636dfe49d70 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f832f7fe640 1 -- 172.21.15.64:0/295767804 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8340041e60 con 0x5636dfe49d70 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 10 client.5364 fetch_fsmap learned FSMap version 15 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5636dfdb4140 con 0x5636dfe49d70 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f832f7fe640 1 -- 172.21.15.64:0/295767804 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636dfdaeaa0 con 0x5636dfe49d70 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f832f7fe640 1 -- 172.21.15.64:0/295767804 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f834003c040 con 0x5636dfe49d70 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 10 client.5364 fetch_fsmap finished waiting for FSMap version 15 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 10 client.5364 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 20 client.5364 populate_metadata read hostname 'smithi064' 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe9a810 0x5636dfe74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 4 client.5364 mds_command: new command op to 5199 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.008+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 --> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636dfdae7c0 con 0x5636dfe9a810 2023-11-24T06:29:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.009+0000 7f834ffff640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe9a810 0x5636dfe74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.009+0000 7f834ffff640 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe9a810 0x5636dfe74460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40863561541d59d2 server_cookie=25f97bef2489ca56 in_seq=0 out_seq=0 2023-11-24T06:29:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.009+0000 7f832f7fe640 10 client.5364 ms_handle_connect on v2:172.21.15.64:6834/2541249514 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f832f7fe640 1 -- 172.21.15.64:0/295767804 <== mds.0 v2:172.21.15.64:6834/2541249514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636dfdae7c0 con 0x5636dfe9a810 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f832f7fe640 10 client.5364 handle_command_reply: tid=0 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 client.5364 shutdown 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe49d70 msgr2=0x5636dfe5fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe49d70 0x5636dfe5fcc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8340036040 tx=0x7f8340004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 shutdown_connections 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe9a810 0x5636dfe74460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4f070 0x5636dfe62380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe9c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 --2- 172.21.15.64:0/295767804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe49d70 0x5636dfe5fcc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f832f7fe640 0 client.5364 ms_handle_reset on v2:172.21.15.64:6834/2541249514 2023-11-24T06:29:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 >> 172.21.15.64:0/295767804 conn(0x5636dfe10c70 msgr2=0x5636dfe48dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 shutdown_connections 2023-11-24T06:29:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.010+0000 7f8359bc3740 1 -- 172.21.15.64:0/295767804 wait complete. 2023-11-24T06:29:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.011+0000 7f8359bc3740 20 client.5364 trim_cache size 0 max 16384 2023-11-24T06:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.028+0000 7f8359bc3740 1 Processor -- start 2023-11-24T06:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.028+0000 7f8359bc3740 1 -- start start 2023-11-24T06:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.029+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe49d70 0x5636dfe93d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.029+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe61a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.029+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dfe61f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.029+0000 7f8359bc3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfb18f20 con 0x5636dfe4f070 2023-11-24T06:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.029+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfdb79a0 con 0x5636dfe49d70 2023-11-24T06:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.029+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636dfea4900 con 0x5636dfe4e7b0 2023-11-24T06:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f8354ce8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe49d70 0x5636dfe93d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f83554e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dfe61f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f8354ce8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe49d70 0x5636dfe93d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:53856/0 (socket says 172.21.15.64:53856) 2023-11-24T06:29:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f8354ce8640 1 -- 172.21.15.64:0/2845688132 learned_addr learned my addr 172.21.15.64:0/2845688132 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f834ffff640 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe61a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f83554e9640 1 -- 172.21.15.64:0/2845688132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 msgr2=0x5636dfe61a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f83554e9640 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe61a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f83554e9640 1 -- 172.21.15.64:0/2845688132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe49d70 msgr2=0x5636dfe93d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f83554e9640 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe49d70 0x5636dfe93d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f83554e9640 1 -- 172.21.15.64:0/2845688132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636dfd51b80 con 0x5636dfe4f070 2023-11-24T06:29:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.030+0000 7f83554e9640 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dfe61f40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8348057570 tx=0x7f8348017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b072d75ac568937d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f83377fe640 1 -- 172.21.15.64:0/2845688132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348056f40 con 0x5636dfe4f070 2023-11-24T06:29:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f83377fe640 1 -- 172.21.15.64:0/2845688132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834805f8e0 con 0x5636dfe4f070 2023-11-24T06:29:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f83377fe640 1 -- 172.21.15.64:0/2845688132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834805f450 con 0x5636dfe4f070 2023-11-24T06:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 -- 172.21.15.64:0/2845688132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 msgr2=0x5636dfe61f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dfe61f40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8348057570 tx=0x7f8348017000 comp rx=0 tx=0).stop 2023-11-24T06:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 -- 172.21.15.64:0/2845688132 shutdown_connections 2023-11-24T06:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dfe61f40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe4e7b0 0x5636dfe61a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 --2- 172.21.15.64:0/2845688132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe49d70 0x5636dfe93d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 -- 172.21.15.64:0/2845688132 >> 172.21.15.64:0/2845688132 conn(0x5636dfe10c70 msgr2=0x5636dfe94b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:18.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.031+0000 7f8359bc3740 1 -- 172.21.15.64:0/2845688132 shutdown_connections 2023-11-24T06:29:18.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.032+0000 7f8359bc3740 1 -- 172.21.15.64:0/2845688132 wait complete. 2023-11-24T06:29:18.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.032+0000 7f8359bc3740 1 -- start start 2023-11-24T06:29:18.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.033+0000 7f8359bc3740 1 Processor -- start 2023-11-24T06:29:18.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 0x5636dfefc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4e7b0 0x5636dfefec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f8359bc3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dff029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f8359bc3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfeff6c0 con 0x5636dfe4f070 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636dfe92700 con 0x5636dfe4e7b0 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f8359bc3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636dff02230 con 0x5636dfe49d70 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f8359bc3740 10 client.0 _ll_register_callbacks cb 0x7f8340009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dff029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dff029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37458/0 (socket says 172.21.15.64:37458) 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 -- 172.21.15.64:0/575456330 learned_addr learned my addr 172.21.15.64:0/575456330 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:18.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 -- 172.21.15.64:0/575456330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 msgr2=0x5636dfefc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 0x5636dfefc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f834ffff640 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4e7b0 0x5636dfefec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 -- 172.21.15.64:0/575456330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4e7b0 msgr2=0x5636dfefec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4e7b0 0x5636dfefec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.034+0000 7f83554e9640 1 -- 172.21.15.64:0/575456330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636dfd7d860 con 0x5636dfe4f070 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f83554e9640 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dff029a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8348082010 tx=0x7f834801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfbba4ca71a0216 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f832f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T06:29:18.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f832f7fe640 1 -- 172.21.15.64:0/575456330 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348056f40 con 0x5636dfe4f070 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f832f7fe640 1 -- 172.21.15.64:0/575456330 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83480533d0 con 0x5636dfe4f070 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8348055980 con 0x5636dfe4f070 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f832f7fe640 1 -- 172.21.15.64:0/575456330 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f834800ce50 con 0x5636dfe4f070 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 10 client.5370 fetch_fsmap learned FSMap version 15 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5636dfd77170 con 0x5636dfe4f070 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f832f7fe640 1 -- 172.21.15.64:0/575456330 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348064020 con 0x5636dfe4f070 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f832f7fe640 1 -- 172.21.15.64:0/575456330 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8348053590 con 0x5636dfe4f070 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 10 client.5370 fetch_fsmap finished waiting for FSMap version 15 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 10 client.5370 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 20 client.5370 populate_metadata read hostname 'smithi064' 2023-11-24T06:29:18.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe48dd0 0x5636dfea70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 4 client.5370 mds_command: new command op to 5199 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.035+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 --> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5636dfab66c0 con 0x5636dfe48dd0 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.036+0000 7f8354ce8640 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe48dd0 0x5636dfea70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.036+0000 7f8354ce8640 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe48dd0 0x5636dfea70b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708d2f36c7fa8698 server_cookie=8500e956e9ad0a79 in_seq=0 out_seq=0 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.036+0000 7f832f7fe640 10 client.5370 ms_handle_connect on v2:172.21.15.64:6834/2541249514 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.036+0000 7f832f7fe640 1 -- 172.21.15.64:0/575456330 <== mds.0 v2:172.21.15.64:6834/2541249514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5636dfab66c0 con 0x5636dfe48dd0 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.036+0000 7f832f7fe640 10 client.5370 handle_command_reply: tid=0 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 client.5370 shutdown 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 msgr2=0x5636dff029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dff029a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8348082010 tx=0x7f834801d040 comp rx=0 tx=0).stop 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 shutdown_connections 2023-11-24T06:29:18.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:6834/2541249514,v1:172.21.15.64:6836/2541249514] conn(0x5636dfe48dd0 0x5636dfea70b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636dfe4f070 0x5636dff029a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5636dfe4e7b0 0x5636dfefec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 --2- 172.21.15.64:0/575456330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5636dfe49d70 0x5636dfefc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f832f7fe640 0 client.5370 ms_handle_reset on v2:172.21.15.64:6834/2541249514 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 >> 172.21.15.64:0/575456330 conn(0x5636dfe10c70 msgr2=0x5636dfe79480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 shutdown_connections 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 1 -- 172.21.15.64:0/575456330 wait complete. 2023-11-24T06:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.037+0000 7f8359bc3740 20 client.5370 trim_cache size 0 max 16384 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "fb91be48-aa54-48b4-b765-8060df323172", 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5199, 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 6.7645309249999999, 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 15, 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 36, 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 36, 2023-11-24T06:29:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 16.570564438000002 2023-11-24T06:29:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f833c06ae40 msgr2=0x7f833c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f833c06ae40 0x7f833c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8340035000 tx=0x7f8340030000 comp rx=0 tx=0).stop 2023-11-24T06:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350165bb0 msgr2=0x7f835015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350165bb0 0x7f835015b2b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f834800c360 tx=0x7f834800c820 comp rx=0 tx=0).stop 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 shutdown_connections 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f833c06ae40 0x7f833c06d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350165bb0 0x7f835015b2b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8350162430 0x7f8350159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 --2- 172.21.15.64:0/2256808887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83500b3650 0x7f835015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 >> 172.21.15.64:0/2256808887 conn(0x7f835006a800 msgr2=0x7f83500b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 shutdown_connections 2023-11-24T06:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.038+0000 7f8355cea640 1 -- 172.21.15.64:0/2256808887 wait complete. 2023-11-24T06:29:18.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:29:18.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.279+0000 7f6fbee7c640 1 Processor -- start 2023-11-24T06:29:18.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.280+0000 7f6fbee7c640 1 -- start start 2023-11-24T06:29:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.281+0000 7f6fbee7c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6fb8143ae0 0x7f6fb80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.281+0000 7f6fbee7c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6fb815e3b0 0x7f6fb80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.281+0000 7f6fbee7c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb80cfc30 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.281+0000 7f6fbee7c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb80b1ee0 con 0x7f6fb8143ae0 2023-11-24T06:29:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.281+0000 7f6fbee7c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb80b2780 con 0x7f6fb815e3b0 2023-11-24T06:29:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.282+0000 7f6fb7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6fb80cf7d0 0x7f6fb815dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59538/0 (socket says 172.21.15.64:59538) 2023-11-24T06:29:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.282+0000 7f6fb7fff640 1 -- 172.21.15.64:0/1845889344 learned_addr learned my addr 172.21.15.64:0/1845889344 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.282+0000 7f6fbcc18640 1 --2- 172.21.15.64:0/1845889344 >> v2:172.21.15.64:3300/0 conn(0x7f6fb8143ae0 0x7f6fb80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.282+0000 7f6fbd419640 1 --2- 172.21.15.64:0/1845889344 >> v2:172.21.15.162:3300/0 conn(0x7f6fb815e3b0 0x7f6fb80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.282+0000 7f6fb77fe640 1 -- 172.21.15.64:0/1845889344 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499156853 0 0) 0x7f6fb80cfc30 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.282+0000 7f6fb77fe640 1 -- 172.21.15.64:0/1845889344 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa8005730 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.283+0000 7f6fbcc18640 1 -- 172.21.15.64:0/1845889344 >> v2:172.21.15.162:3300/0 conn(0x7f6fb815e3b0 msgr2=0x7f6fb80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.283+0000 7f6fbcc18640 1 --2- 172.21.15.64:0/1845889344 >> v2:172.21.15.162:3300/0 conn(0x7f6fb815e3b0 0x7f6fb80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.283+0000 7f6fbcc18640 1 -- 172.21.15.64:0/1845889344 >> v1:172.21.15.64:6789/0 conn(0x7f6fb80cf7d0 legacy=0x7f6fb815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.283+0000 7f6fbcc18640 1 -- 172.21.15.64:0/1845889344 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb815e990 con 0x7f6fb8143ae0 2023-11-24T06:29:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.284+0000 7f6fbcc18640 1 --2- 172.21.15.64:0/1845889344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8143ae0 0x7f6fb80cf290 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6fb806e4f0 tx=0x7f6fac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdecd43f1715af75 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.284+0000 7f6fb77fe640 1 -- 172.21.15.64:0/1845889344 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac031830 con 0x7f6fb8143ae0 2023-11-24T06:29:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.284+0000 7f6fb77fe640 1 -- 172.21.15.64:0/1845889344 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac031990 con 0x7f6fb8143ae0 2023-11-24T06:29:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.284+0000 7f6fb77fe640 1 -- 172.21.15.64:0/1845889344 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac031d20 con 0x7f6fb8143ae0 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 -- 172.21.15.64:0/1845889344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8143ae0 msgr2=0x7f6fb80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/1845889344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8143ae0 0x7f6fb80cf290 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6fb806e4f0 tx=0x7f6fac02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 -- 172.21.15.64:0/1845889344 shutdown_connections 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/1845889344 >> v2:172.21.15.162:3300/0 conn(0x7f6fb815e3b0 0x7f6fb80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/1845889344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8143ae0 0x7f6fb80cf290 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 -- 172.21.15.64:0/1845889344 >> 172.21.15.64:0/1845889344 conn(0x7f6fb806a800 msgr2=0x7f6fb806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 -- 172.21.15.64:0/1845889344 shutdown_connections 2023-11-24T06:29:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.285+0000 7f6fbee7c640 1 -- 172.21.15.64:0/1845889344 wait complete. 2023-11-24T06:29:18.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.287+0000 7f6fbee7c640 1 Processor -- start 2023-11-24T06:29:18.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.287+0000 7f6fbee7c640 1 -- start start 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbee7c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb80cf7d0 0x7f6fb81376e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbee7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb8143ae0 0x7f6fb8140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbee7c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb815e3b0 0x7f6fb813f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbee7c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb813fc20 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbee7c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb813ff00 con 0x7f6fb815e3b0 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbee7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81401e0 con 0x7f6fb8143ae0 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbcc18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb80cf7d0 0x7f6fb81376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbcc18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb80cf7d0 0x7f6fb81376e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37484/0 (socket says 172.21.15.64:37484) 2023-11-24T06:29:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.288+0000 7f6fbcc18640 1 -- 172.21.15.64:0/2620491135 learned_addr learned my addr 172.21.15.64:0/2620491135 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fb7fff640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb8143ae0 0x7f6fb8140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbd419640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb815e3b0 0x7f6fb813f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbcc18640 1 -- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb8143ae0 msgr2=0x7f6fb8140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbcc18640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb8143ae0 0x7f6fb8140cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbcc18640 1 -- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb815e3b0 msgr2=0x7f6fb813f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbcc18640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb815e3b0 0x7f6fb813f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbcc18640 1 -- 172.21.15.64:0/2620491135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac009400 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fb7fff640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb8143ae0 0x7f6fb8140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbcc18640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb80cf7d0 0x7f6fb81376e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009890 tx=0x7f6fac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fb57fa640 1 -- 172.21.15.64:0/2620491135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac03a630 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fb57fa640 1 -- 172.21.15.64:0/2620491135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac03a7b0 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fb57fa640 1 -- 172.21.15.64:0/2620491135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac03ab60 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.289+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb81404c0 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.290+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb8137c20 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.291+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb80675d0 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.292+0000 7f6fb57fa640 1 -- 172.21.15.64:0/2620491135 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fac03acc0 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.293+0000 7f6fb57fa640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6f9c06ae40 0x7f6f9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.293+0000 7f6fb57fa640 1 -- 172.21.15.64:0/2620491135 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fac0ab1c0 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.293+0000 7f6fb7fff640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6f9c06ae40 0x7f6f9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.294+0000 7f6fb7fff640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6f9c06ae40 0x7f6f9c06d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8141990 tx=0x7f6fa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.297+0000 7f6fb57fa640 1 -- 172.21.15.64:0/2620491135 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fac076390 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.532+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6fb8067e40 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.533+0000 7f6fb57fa640 1 -- 172.21.15.64:0/2620491135 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6fac0799b0 con 0x7f6fb80cf7d0 2023-11-24T06:29:18.536 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:29:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6f9c06ae40 msgr2=0x7f6f9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6f9c06ae40 0x7f6f9c06d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8141990 tx=0x7f6fa4009040 comp rx=0 tx=0).stop 2023-11-24T06:29:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb80cf7d0 msgr2=0x7f6fb81376e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb80cf7d0 0x7f6fb81376e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009890 tx=0x7f6fac0046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 shutdown_connections 2023-11-24T06:29:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f6f9c06ae40 0x7f6f9c06d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb815e3b0 0x7f6fb813f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb8143ae0 0x7f6fb8140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 --2- 172.21.15.64:0/2620491135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb80cf7d0 0x7f6fb81376e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 >> 172.21.15.64:0/2620491135 conn(0x7f6fb806a800 msgr2=0x7f6fb815b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 shutdown_connections 2023-11-24T06:29:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:18.536+0000 7f6fbee7c640 1 -- 172.21.15.64:0/2620491135 wait complete. 2023-11-24T06:29:18.556 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:29:18.556 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:29:18.556 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T06:29:18.557 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:29:18.557 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:18.557 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:29:18.557 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:29:18.557 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:29:18.557 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:29:18.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.557 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T06:29:18.607 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:29:18.608 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:29:18.608 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:29:18.608 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:18.608 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T06:29:18.608 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:18.608 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:18.609 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:18.609 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T06:29:18.609 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:18.609 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:18.609 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T06:29:18.609 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42236sec preferred_lft 42236sec 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:18.610 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:29:18.611 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 72:68:86:cc:50:a0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:18.611 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:29:18.611 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:18.611 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::64a0:9fff:fe5c:508c/64 scope link 2023-11-24T06:29:18.611 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:18.612 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:18.612 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 92:32:08:7d:09:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:18.612 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9032:8ff:fe7d:9ea/64 scope link 2023-11-24T06:29:18.612 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:18.612 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:18.613 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 72:68:86:cc:50:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:18.613 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::7068:86ff:fecc:50a0/64 scope link 2023-11-24T06:29:18.613 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:18.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.614 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:29:18.705 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T06:29:18.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.710 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T06:29:18.770 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T06:29:18.770 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:29:18.770 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:29:18.770 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:29:18.771 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:29:18.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.771 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T06:29:18.771 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T06:29:18.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:29:18.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:29:18.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:29:18.771 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T06:29:18.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.970 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T06:29:19.023 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:29:19.023 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:29:19.024 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:29:19.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:19.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T06:29:19.122 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:29:19.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:19.123 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:19.179 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:29:19.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:19.179 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:19.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T06:29:19.339 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:19.399 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T06:29:19.399 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T06:29:19.399 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T06:29:19.399 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T06:29:19.399 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T06:29:19.399 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:19.400 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:19.401 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:19.402 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:19.402 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:19.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:19.403 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T06:29:19.462 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:29:19.462 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:29:19.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T06:29:19.548 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T06:29:19.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:19.593 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:19.595 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:19.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:19.596 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T06:29:19.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T06:29:19.656+0000 7f7184c47480 -1 init, newargv = 0x563ba25e3980 newargc=15 2023-11-24T06:29:19.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[101834]: starting ceph client 2023-11-24T06:29:19.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[101834]: starting fuse 2023-11-24T06:29:19.687 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T06:29:19.687 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:29:19.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T06:29:19.688 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T06:29:19.689 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T06:29:19.909 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.101834.asok 2023-11-24T06:29:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:29:19.927 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.101834.asok 2023-11-24T06:29:19.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:19.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.101834.asok status 2023-11-24T06:29:20.179 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:20.179 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T06:29:20.179 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:29:20.179 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:29:20.179 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T06:29:20.179 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "101834", 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5385, 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T06:29:20.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5385 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3697209589 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:20.181 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3697209589 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5385 192.168.0.1:0/3697209589", 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3697209589", 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 15, 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 36, 2023-11-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:29:20.183 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T06:29:20.183 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T06:29:20.183 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:20.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:20.194 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T06:29:20.194 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:20.253 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T06:29:20.253 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:20.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:20.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:20.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:29:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.579+0000 7f16d65cb640 1 Processor -- start 2023-11-24T06:29:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.579+0000 7f16d65cb640 1 -- start start 2023-11-24T06:29:20.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.580+0000 7f16d65cb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 0x7f16d0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.580+0000 7f16d65cb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d0070e50 con 0x7f16d00ccc10 2023-11-24T06:29:20.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.581+0000 7f16d65cb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d0144020 con 0x7f16d00717d0 2023-11-24T06:29:20.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.581+0000 7f16d65cb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00723c0 con 0x7f16d0143c40 2023-11-24T06:29:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.581+0000 7f16d5dca640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f16d00ccc10 0x7f16d006d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59554/0 (socket says 172.21.15.64:59554) 2023-11-24T06:29:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.581+0000 7f16d5dca640 1 -- 172.21.15.64:0/1754232119 learned_addr learned my addr 172.21.15.64:0/1754232119 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.581+0000 7f16d55c9640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 0x7f16d0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.581+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455008324 0 0) 0x7f16d0070e50 con 0x7f16d00ccc10 2023-11-24T06:29:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.581+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c0003520 con 0x7f16d00ccc10 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2016438063 0 0) 0x7f16c0003520 con 0x7f16d00ccc10 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 msgr2=0x7f16d0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 0x7f16d0071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 >> v1:172.21.15.162:6789/0 conn(0x7f16d00717d0 legacy=0x7f16d00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0071e00 con 0x7f16d00ccc10 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f16c8001670 con 0x7f16d00ccc10 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 >> v1:172.21.15.64:6789/0 conn(0x7f16d00ccc10 legacy=0x7f16d006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c00039a0 0x7f16c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 0x7f16c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00723c0 con 0x7f16c0004d00 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c000bab0 con 0x7f16c00039a0 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c000bc70 con 0x7f16d0143c40 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.582+0000 7f16d5dca640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.583+0000 7f16d55c9640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c00039a0 0x7f16c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.583+0000 7f16d4dc8640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 0x7f16c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.583+0000 7f16d5dca640 1 -- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 msgr2=0x7f16c00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.583+0000 7f16d5dca640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d0143c40 0x7f16c00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.583+0000 7f16d5dca640 1 -- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c00039a0 msgr2=0x7f16c0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.583+0000 7f16d5dca640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16c00039a0 0x7f16c0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.583+0000 7f16d5dca640 1 -- 172.21.15.64:0/1754232119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0003520 con 0x7f16c0004d00 2023-11-24T06:29:20.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.584+0000 7f16d5dca640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16c0005180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c240 tx=0x7f16c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df16492ce3efdeb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.584+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8005ad0 con 0x7f16c0004d00 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.584+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c8005c30 con 0x7f16c0004d00 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.584+0000 7f16cffff640 1 -- 172.21.15.64:0/1754232119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c802c8b0 con 0x7f16c0004d00 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.585+0000 7f16d65cb640 1 -- 172.21.15.64:0/1754232119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 msgr2=0x7f16c0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.585+0000 7f16d65cb640 1 --2- 172.21.15.64:0/1754232119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16c0005180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c240 tx=0x7f16c00057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.585+0000 7f16d5dca640 1 -- 172.21.15.64:0/1754232119 reap_dead start 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.585+0000 7f16d65cb640 1 -- 172.21.15.64:0/1754232119 shutdown_connections 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.585+0000 7f16d65cb640 1 -- 172.21.15.64:0/1754232119 >> 172.21.15.64:0/1754232119 conn(0x7f16d006a800 msgr2=0x7f16d006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.585+0000 7f16d65cb640 1 -- 172.21.15.64:0/1754232119 shutdown_connections 2023-11-24T06:29:20.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.585+0000 7f16d65cb640 1 -- 172.21.15.64:0/1754232119 wait complete. 2023-11-24T06:29:20.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.588+0000 7f16d65cb640 1 Processor -- start 2023-11-24T06:29:20.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.588+0000 7f16d65cb640 1 -- start start 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.589+0000 7f16d65cb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16d0154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.589+0000 7f16d65cb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d01589a0 0x7f16d01e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.589+0000 7f16d65cb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d01e7340 0x7f16d0137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.589+0000 7f16d65cb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0138f70 con 0x7f16c0004d00 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.589+0000 7f16d65cb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d01e77c0 con 0x7f16d01e7340 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.589+0000 7f16d65cb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0158f40 con 0x7f16d01589a0 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16d0154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16d0154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37502/0 (socket says 172.21.15.64:37502) 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 -- 172.21.15.64:0/1307797936 learned_addr learned my addr 172.21.15.64:0/1307797936 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d4dc8640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d01589a0 0x7f16d01e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d5dca640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d01e7340 0x7f16d0137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 -- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d01589a0 msgr2=0x7f16d01e6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d01589a0 0x7f16d01e6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 -- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d01e7340 msgr2=0x7f16d0137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d01e7340 0x7f16d0137ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d55c9640 1 -- 172.21.15.64:0/1307797936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0159220 con 0x7f16c0004d00 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d4dc8640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d01589a0 0x7f16d01e6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.590+0000 7f16d5dca640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d01e7340 0x7f16d0137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.591+0000 7f16d55c9640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16d0154c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007f60 tx=0x7f16c4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.591+0000 7f16cdffb640 1 -- 172.21.15.64:0/1307797936 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c400e030 con 0x7f16c0004d00 2023-11-24T06:29:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.591+0000 7f16cdffb640 1 -- 172.21.15.64:0/1307797936 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c4016470 con 0x7f16c0004d00 2023-11-24T06:29:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.591+0000 7f16cdffb640 1 -- 172.21.15.64:0/1307797936 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4016820 con 0x7f16c0004d00 2023-11-24T06:29:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.591+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d0159500 con 0x7f16c0004d00 2023-11-24T06:29:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.591+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d013b670 con 0x7f16c0004d00 2023-11-24T06:29:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.592+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d0067450 con 0x7f16c0004d00 2023-11-24T06:29:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.594+0000 7f16cdffb640 1 -- 172.21.15.64:0/1307797936 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f16c4016b90 con 0x7f16c0004d00 2023-11-24T06:29:20.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.595+0000 7f16cdffb640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16b406ae40 0x7f16b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.595+0000 7f16cdffb640 1 -- 172.21.15.64:0/1307797936 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16c400f070 con 0x7f16c0004d00 2023-11-24T06:29:20.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.595+0000 7f16d4dc8640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16b406ae40 0x7f16b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.596+0000 7f16d4dc8640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16b406ae40 0x7f16b406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f16d01f4370 tx=0x7f16bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.599+0000 7f16cdffb640 1 -- 172.21.15.64:0/1307797936 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16c4055890 con 0x7f16c0004d00 2023-11-24T06:29:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.835+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f16d00ceb30 con 0x7f16c0004d00 2023-11-24T06:29:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.839+0000 7f16cdffb640 1 -- 172.21.15.64:0/1307797936 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f16c4058eb0 con 0x7f16c0004d00 2023-11-24T06:29:20.842 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:29:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16b406ae40 msgr2=0x7f16b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16b406ae40 0x7f16b406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f16d01f4370 tx=0x7f16bc009040 comp rx=0 tx=0).stop 2023-11-24T06:29:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 msgr2=0x7f16d0154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16d0154c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007f60 tx=0x7f16c4007f90 comp rx=0 tx=0).stop 2023-11-24T06:29:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 shutdown_connections 2023-11-24T06:29:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f16b406ae40 0x7f16b406d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d01e7340 0x7f16d0137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16d01589a0 0x7f16d01e6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 --2- 172.21.15.64:0/1307797936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c0004d00 0x7f16d0154c80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 >> 172.21.15.64:0/1307797936 conn(0x7f16d006a800 msgr2=0x7f16d013eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 shutdown_connections 2023-11-24T06:29:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:20.842+0000 7f16d65cb640 1 -- 172.21.15.64:0/1307797936 wait complete. 2023-11-24T06:29:20.864 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:29:20.865 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:29:20.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:20.866 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42254sec preferred_lft 42254sec 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:29:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 72:30:be:f9:3a:3a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::5c20:50ff:fe3d:265e/64 scope link 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether ea:a0:b9:23:79:12 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::e8a0:b9ff:fe23:7912/64 scope link 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:20.930 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 72:30:be:f9:3a:3a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::7030:beff:fef9:3a3a/64 scope link 2023-11-24T06:29:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:20.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:20.931 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:29:21.023 INFO:teuthology.orchestra.run.smithi162.stdout:1 2023-11-24T06:29:21.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:21.028 DEBUG:teuthology.orchestra.run.smithi162:> route 2023-11-24T06:29:21.079 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2023-11-24T06:29:21.079 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:29:21.080 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:29:21.080 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:29:21.080 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:29:21.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:21.080 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-11-24T06:29:21.080 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-11-24T06:29:21.080 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:29:21.080 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:29:21.080 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:29:21.081 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-11-24T06:29:21.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:21.257 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-11-24T06:29:21.307 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:29:21.307 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:29:21.307 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:29:21.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:21.308 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2023-11-24T06:29:21.403 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:29:21.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:21.403 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:21.457 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:29:21.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:21.457 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:21.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-11-24T06:29:21.633 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:21.686 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-11-24T06:29:21.686 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-11-24T06:29:21.686 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-11-24T06:29:21.686 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-11-24T06:29:21.686 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-11-24T06:29:21.686 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-11-24T06:29:21.686 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-11-24T06:29:21.687 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:29:21.688 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-11-24T06:29:21.689 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:21.689 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:21.689 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:21.689 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:21.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:21.690 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-11-24T06:29:21.736 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:29:21.736 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:29:21.785 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-11-24T06:29:21.866 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-11-24T06:29:21.910 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-11-24T06:29:21.911 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-11-24T06:29:21.911 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-11-24T06:29:21.911 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-11-24T06:29:21.911 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:21.911 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-11-24T06:29:21.911 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-11-24T06:29:21.912 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-11-24T06:29:21.912 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-11-24T06:29:21.912 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:21.912 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-11-24T06:29:21.912 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:21.912 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:21.913 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:21.913 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:21.913 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-11-24T06:29:21.913 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-11-24T06:29:21.913 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:29:21.913 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:29:21.914 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:29:21.914 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:29:21.914 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-11-24T06:29:21.914 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:21.914 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:21.915 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:21.915 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:21.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:21.916 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-11-24T06:29:21.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:2023-11-24T06:29:21.971+0000 7f71c7d33480 -1 init, newargv = 0x55cb6ef56930 newargc=15 2023-11-24T06:29:21.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[95590]: starting ceph client 2023-11-24T06:29:21.987 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[95590]: starting fuse 2023-11-24T06:29:22.005 INFO:teuthology.orchestra.run.smithi162.stdout:41 2023-11-24T06:29:22.005 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:29:22.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> import re 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> import os 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:29:22.006 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:29:22.007 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2023-11-24T06:29:22.007 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2023-11-24T06:29:22.007 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2023-11-24T06:29:22.007 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2023-11-24T06:29:22.007 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:29:22.008 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2023-11-24T06:29:22.008 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:29:22.008 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:29:22.008 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:29:22.008 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2023-11-24T06:29:22.008 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2023-11-24T06:29:22.009 DEBUG:teuthology.orchestra.run.smithi162:> return f 2023-11-24T06:29:22.009 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:29:22.009 DEBUG:teuthology.orchestra.run.smithi162:> 2023-11-24T06:29:22.009 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.1")) 2023-11-24T06:29:22.010 DEBUG:teuthology.orchestra.run.smithi162:> ' 2023-11-24T06:29:22.193 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.1.95590.asok 2023-11-24T06:29:22.197 INFO:teuthology.orchestra.run.smithi162.stderr:2023-11-24T06:29:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:29:22.210 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95590.asok 2023-11-24T06:29:22.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:22.210 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95590.asok status 2023-11-24T06:29:22.426 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-11-24T06:29:22.427 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2023-11-24T06:29:22.427 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:29:22.427 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:29:22.427 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "1", 2023-11-24T06:29:22.428 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2023-11-24T06:29:22.428 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:29:22.428 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "95590", 2023-11-24T06:29:22.428 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2023-11-24T06:29:22.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:29:22.428 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2023-11-24T06:29:22.429 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2023-11-24T06:29:22.429 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 5059, 2023-11-24T06:29:22.429 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2023-11-24T06:29:22.429 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2023-11-24T06:29:22.429 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 5059 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 2778976974 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-11-24T06:29:22.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 2778976974 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.5059 192.168.0.1:0/2778976974", 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.1:0/2778976974", 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 15, 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 36, 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2023-11-24T06:29:22.431 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-11-24T06:29:22.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:22.442 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2023-11-24T06:29:22.442 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:22.486 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2023-11-24T06:29:22.486 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:22.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:22.487 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:22.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:29:22.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.852+0000 7efd91200640 1 Processor -- start 2023-11-24T06:29:22.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.852+0000 7efd91200640 1 -- start start 2023-11-24T06:29:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.854+0000 7efd91200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efd8c071660 0x7efd8c15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.854+0000 7efd91200640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd8c071a40 0x7efd8c13d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.854+0000 7efd91200640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c0cebf0 con 0x7efd8c06f2a0 2023-11-24T06:29:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.854+0000 7efd91200640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c15e2a0 con 0x7efd8c071a40 2023-11-24T06:29:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.854+0000 7efd91200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c15e580 con 0x7efd8c071660 2023-11-24T06:29:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.854+0000 7efd8a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd8c071a40 0x7efd8c13d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8b59e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efd8c06f2a0 0x7efd8c0cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57580/0 (socket says 172.21.15.64:57580) 2023-11-24T06:29:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8b59e640 1 -- 172.21.15.64:0/3659381906 learned_addr learned my addr 172.21.15.64:0/3659381906 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8ad9d640 1 --2- 172.21.15.64:0/3659381906 >> v2:172.21.15.162:3300/0 conn(0x7efd8c071660 0x7efd8c15dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8a59c640 1 -- 172.21.15.64:0/3659381906 >> v2:172.21.15.162:3300/0 conn(0x7efd8c071660 msgr2=0x7efd8c15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8a59c640 1 --2- 172.21.15.64:0/3659381906 >> v2:172.21.15.162:3300/0 conn(0x7efd8c071660 0x7efd8c15dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8a59c640 1 -- 172.21.15.64:0/3659381906 >> v1:172.21.15.64:6789/0 conn(0x7efd8c06f2a0 legacy=0x7efd8c0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8a59c640 1 -- 172.21.15.64:0/3659381906 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c15e860 con 0x7efd8c071a40 2023-11-24T06:29:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.855+0000 7efd8ad9d640 1 --2- 172.21.15.64:0/3659381906 >> v2:172.21.15.162:3300/0 conn(0x7efd8c071660 0x7efd8c15dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.857+0000 7efd8a59c640 1 --2- 172.21.15.64:0/3659381906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071a40 0x7efd8c13d840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd8c06ffa0 tx=0x7efd7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd38df2accb73f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.857+0000 7efd89d9b640 1 -- 172.21.15.64:0/3659381906 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78031830 con 0x7efd8c071a40 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.857+0000 7efd89d9b640 1 -- 172.21.15.64:0/3659381906 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78031990 con 0x7efd8c071a40 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.857+0000 7efd89d9b640 1 -- 172.21.15.64:0/3659381906 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78031d20 con 0x7efd8c071a40 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 -- 172.21.15.64:0/3659381906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071a40 msgr2=0x7efd8c13d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 --2- 172.21.15.64:0/3659381906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071a40 0x7efd8c13d840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd8c06ffa0 tx=0x7efd7802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 -- 172.21.15.64:0/3659381906 shutdown_connections 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 --2- 172.21.15.64:0/3659381906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071a40 0x7efd8c13d840 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 --2- 172.21.15.64:0/3659381906 >> v2:172.21.15.162:3300/0 conn(0x7efd8c071660 0x7efd8c15dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 -- 172.21.15.64:0/3659381906 >> 172.21.15.64:0/3659381906 conn(0x7efd8c0b1dd0 msgr2=0x7efd8c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 -- 172.21.15.64:0/3659381906 shutdown_connections 2023-11-24T06:29:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.858+0000 7efd91200640 1 -- 172.21.15.64:0/3659381906 wait complete. 2023-11-24T06:29:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.860+0000 7efd91200640 1 Processor -- start 2023-11-24T06:29:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.860+0000 7efd91200640 1 -- start start 2023-11-24T06:29:22.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd91200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd8c06f2a0 0x7efd8c0ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd91200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071660 0x7efd8c141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd91200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd8c071a40 0x7efd8c1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd91200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c1f3d00 con 0x7efd8c071660 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd91200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0adad0 con 0x7efd8c071a40 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd91200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0addb0 con 0x7efd8c06f2a0 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071660 0x7efd8c141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071660 0x7efd8c141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37336/0 (socket says 172.21.15.64:37336) 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd8a59c640 1 -- 172.21.15.64:0/1827484594 learned_addr learned my addr 172.21.15.64:0/1827484594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.862+0000 7efd8ad9d640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd8c06f2a0 0x7efd8c0ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8b59e640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd8c071a40 0x7efd8c1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8a59c640 1 -- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd8c06f2a0 msgr2=0x7efd8c0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8a59c640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd8c06f2a0 0x7efd8c0ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8a59c640 1 -- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd8c071a40 msgr2=0x7efd8c1f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8a59c640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd8c071a40 0x7efd8c1f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8a59c640 1 -- 172.21.15.64:0/1827484594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78009400 con 0x7efd8c071660 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8ad9d640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd8c06f2a0 0x7efd8c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8b59e640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd8c071a40 0x7efd8c1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd8a59c640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071660 0x7efd8c141d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efd78000c00 tx=0x7efd780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd837fe640 1 -- 172.21.15.64:0/1827484594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7803aa60 con 0x7efd8c071660 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd837fe640 1 -- 172.21.15.64:0/1827484594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7803abe0 con 0x7efd8c071660 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd837fe640 1 -- 172.21.15.64:0/1827484594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd780443f0 con 0x7efd8c071660 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.863+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c0ae090 con 0x7efd8c071660 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.864+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c0ae650 con 0x7efd8c071660 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.865+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c0ae840 con 0x7efd8c071660 2023-11-24T06:29:22.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.866+0000 7efd837fe640 1 -- 172.21.15.64:0/1827484594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd7804d450 con 0x7efd8c071660 2023-11-24T06:29:22.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.867+0000 7efd837fe640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efd7406ae40 0x7efd7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.867+0000 7efd837fe640 1 -- 172.21.15.64:0/1827484594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd780ab710 con 0x7efd8c071660 2023-11-24T06:29:22.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.867+0000 7efd8ad9d640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efd7406ae40 0x7efd7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.868+0000 7efd8ad9d640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efd7406ae40 0x7efd7406d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006fd0 tx=0x7efd7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:22.872+0000 7efd837fe640 1 -- 172.21.15.64:0/1827484594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd78076850 con 0x7efd8c071660 2023-11-24T06:29:23.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.107+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efd8c0675d0 con 0x7efd8c071660 2023-11-24T06:29:23.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.859+0000 7efd837fe640 1 -- 172.21.15.64:0/1827484594 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 136+0+0 (secure 0 0 0) 0x7efd780350c0 con 0x7efd8c071660 2023-11-24T06:29:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efd7406ae40 msgr2=0x7efd7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efd7406ae40 0x7efd7406d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006fd0 tx=0x7efd7c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071660 msgr2=0x7efd8c141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071660 0x7efd8c141d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efd78000c00 tx=0x7efd780046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 shutdown_connections 2023-11-24T06:29:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7efd7406ae40 0x7efd7406d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd8c071a40 0x7efd8c1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c071660 0x7efd8c141d80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 --2- 172.21.15.64:0/1827484594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd8c06f2a0 0x7efd8c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.863+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 >> 172.21.15.64:0/1827484594 conn(0x7efd8c0b1dd0 msgr2=0x7efd8c13dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.864+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 shutdown_connections 2023-11-24T06:29:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:23.864+0000 7efd91200640 1 -- 172.21.15.64:0/1827484594 wait complete. 2023-11-24T06:29:23.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:29:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.115+0000 7f2d57fff640 1 Processor -- start 2023-11-24T06:29:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.115+0000 7f2d57fff640 1 -- start start 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.117+0000 7f2d57fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d580cc500 0x7f2d580cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.117+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 0x7f2d5806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.117+0000 7f2d57fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d58070e50 con 0x7f2d58143c40 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.117+0000 7f2d57fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d580723c0 con 0x7f2d580cc500 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.117+0000 7f2d57fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580cced0 con 0x7f2d5806d160 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.117+0000 7f2d56ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2d58143c40 0x7f2d58071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57586/0 (socket says 172.21.15.64:57586) 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.117+0000 7f2d56ffd640 1 -- 172.21.15.64:0/2293340040 learned_addr learned my addr 172.21.15.64:0/2293340040 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d4ffff640 1 --2- 172.21.15.64:0/2293340040 >> v2:172.21.15.162:3300/0 conn(0x7f2d580cc500 0x7f2d580cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d577fe640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 0x7f2d5806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278533979 0 0) 0x7f2d58070e50 con 0x7f2d58143c40 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d44003520 con 0x7f2d58143c40 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438496655 0 0) 0x7f2d44003520 con 0x7f2d58143c40 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 msgr2=0x7f2d5806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 0x7f2d5806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 >> v2:172.21.15.162:3300/0 conn(0x7f2d580cc500 msgr2=0x7f2d580cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 --2- 172.21.15.64:0/2293340040 >> v2:172.21.15.162:3300/0 conn(0x7f2d580cc500 0x7f2d580cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58144020 con 0x7f2d58143c40 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f2d48001690 con 0x7f2d58143c40 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 >> v1:172.21.15.64:6789/0 conn(0x7f2d58143c40 legacy=0x7f2d58071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d44003970 0x7f2d44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.118+0000 7f2d567fc640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d440047c0 0x7f2d44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d567fc640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 0x7f2d44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580723c0 con 0x7f2d44003970 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580cced0 con 0x7f2d440047c0 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4400b9b0 con 0x7f2d5806d160 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d44003970 0x7f2d44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d577fe640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 0x7f2d44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d56ffd640 1 -- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 msgr2=0x7f2d44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d5806d160 0x7f2d44005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d56ffd640 1 -- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d440047c0 msgr2=0x7f2d44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d440047c0 0x7f2d44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.119+0000 7f2d56ffd640 1 -- 172.21.15.64:0/2293340040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44003520 con 0x7f2d44003970 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.120+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d44003970 0x7f2d44003d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2d48007fd0 tx=0x7f2d440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b27d2623a3f12d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48005ad0 con 0x7f2d44003970 2023-11-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d48005c30 con 0x7f2d44003970 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d567fc640 1 -- 172.21.15.64:0/2293340040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4802c8c0 con 0x7f2d44003970 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d57fff640 1 -- 172.21.15.64:0/2293340040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d44003970 msgr2=0x7f2d44003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d57fff640 1 --2- 172.21.15.64:0/2293340040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d44003970 0x7f2d44003d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2d48007fd0 tx=0x7f2d440057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d577fe640 1 -- 172.21.15.64:0/2293340040 reap_dead start 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d57fff640 1 -- 172.21.15.64:0/2293340040 shutdown_connections 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.121+0000 7f2d57fff640 1 -- 172.21.15.64:0/2293340040 >> 172.21.15.64:0/2293340040 conn(0x7f2d5806a800 msgr2=0x7f2d5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.122+0000 7f2d57fff640 1 -- 172.21.15.64:0/2293340040 shutdown_connections 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.122+0000 7f2d57fff640 1 -- 172.21.15.64:0/2293340040 wait complete. 2023-11-24T06:29:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.123+0000 7f2d57fff640 1 Processor -- start 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.124+0000 7f2d57fff640 1 -- start start 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.124+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d44003970 0x7f2d5815a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.124+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580aa550 0x7f2d580ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580aceb0 0x7f2d580ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d57fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580aca20 con 0x7f2d580aceb0 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d57fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580abe10 con 0x7f2d580aa550 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d57fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580ac0f0 con 0x7f2d44003970 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580aceb0 0x7f2d580ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580aceb0 0x7f2d580ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37350/0 (socket says 172.21.15.64:37350) 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 -- 172.21.15.64:0/3925907200 learned_addr learned my addr 172.21.15.64:0/3925907200 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 -- 172.21.15.64:0/3925907200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d44003970 msgr2=0x7f2d5815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d44003970 0x7f2d5815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 -- 172.21.15.64:0/3925907200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580aa550 msgr2=0x7f2d580ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580aa550 0x7f2d580ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.125+0000 7f2d577fe640 1 -- 172.21.15.64:0/3925907200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d580ac3d0 con 0x7f2d580aceb0 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.126+0000 7f2d577fe640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580aceb0 0x7f2d580ad340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d58144ca0 tx=0x7f2d5000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.126+0000 7f2d54ff9640 1 -- 172.21.15.64:0/3925907200 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50017400 con 0x7f2d580aceb0 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.126+0000 7f2d54ff9640 1 -- 172.21.15.64:0/3925907200 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d50011040 con 0x7f2d580aceb0 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.126+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d580ac6b0 con 0x7f2d580aceb0 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.126+0000 7f2d54ff9640 1 -- 172.21.15.64:0/3925907200 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500177c0 con 0x7f2d580aceb0 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.126+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d580abbc0 con 0x7f2d580aceb0 2023-11-24T06:29:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.127+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d58067450 con 0x7f2d580aceb0 2023-11-24T06:29:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.128+0000 7f2d54ff9640 1 -- 172.21.15.64:0/3925907200 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d50017920 con 0x7f2d580aceb0 2023-11-24T06:29:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.129+0000 7f2d54ff9640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3806ae70 0x7f2d3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.129+0000 7f2d54ff9640 1 -- 172.21.15.64:0/3925907200 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d50089c00 con 0x7f2d580aceb0 2023-11-24T06:29:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.129+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3806ae70 0x7f2d3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.130+0000 7f2d56ffd640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3806ae70 0x7f2d3806d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d48034000 tx=0x7f2d48007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:24.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.134+0000 7f2d54ff9640 1 -- 172.21.15.64:0/3925907200 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d50054b80 con 0x7f2d580aceb0 2023-11-24T06:29:24.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.354+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d580ac260 con 0x7f2d3806ae70 2023-11-24T06:29:24.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.357+0000 7f2d54ff9640 1 -- 172.21.15.64:0/3925907200 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2d580ac260 con 0x7f2d3806ae70 2023-11-24T06:29:24.361 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T06:29:24.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:24.361 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-11-24T06:29:24.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:24.362 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3806ae70 msgr2=0x7f2d3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3806ae70 0x7f2d3806d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d48034000 tx=0x7f2d48007000 comp rx=0 tx=0).stop 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580aceb0 msgr2=0x7f2d580ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580aceb0 0x7f2d580ad340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d58144ca0 tx=0x7f2d5000f4a0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 shutdown_connections 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f2d3806ae70 0x7f2d3806d330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d580aceb0 0x7f2d580ad340 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580aa550 0x7f2d580ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 --2- 172.21.15.64:0/3925907200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d44003970 0x7f2d5815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 >> 172.21.15.64:0/3925907200 conn(0x7f2d5806a800 msgr2=0x7f2d580afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 shutdown_connections 2023-11-24T06:29:24.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.361+0000 7f2d57fff640 1 -- 172.21.15.64:0/3925907200 wait complete. 2023-11-24T06:29:24.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:29:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.608+0000 7f041328c640 1 Processor -- start 2023-11-24T06:29:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.609+0000 7f041328c640 1 -- start start 2023-11-24T06:29:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.610+0000 7f041328c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f040c146850 0x7f040c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.610+0000 7f041328c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c071b10 0x7f040c0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.610+0000 7f041328c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040c06d480 con 0x7f040c0adb00 2023-11-24T06:29:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.610+0000 7f041328c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c071fa0 con 0x7f040c146850 2023-11-24T06:29:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.610+0000 7f041328c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0b3490 con 0x7f040c071b10 2023-11-24T06:29:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.611+0000 7f041228a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f040c146850 0x7f040c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.611+0000 7f0411a89640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c071b10 0x7f040c0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.611+0000 7f0412a8b640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f040c0adb00 0x7f040c0700f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:60622/0 (socket says 172.21.15.64:60622) 2023-11-24T06:29:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.611+0000 7f0412a8b640 1 -- 172.21.15.64:0/1415803391 learned_addr learned my addr 172.21.15.64:0/1415803391 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.612+0000 7f041228a640 1 -- 172.21.15.64:0/1415803391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c071b10 msgr2=0x7f040c0ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.612+0000 7f041228a640 1 --2- 172.21.15.64:0/1415803391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c071b10 0x7f040c0ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.612+0000 7f041228a640 1 -- 172.21.15.64:0/1415803391 >> v1:172.21.15.162:6789/0 conn(0x7f040c0adb00 legacy=0x7f040c0700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.612+0000 7f041228a640 1 -- 172.21.15.64:0/1415803391 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0b3770 con 0x7f040c146850 2023-11-24T06:29:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.613+0000 7f041228a640 1 --2- 172.21.15.64:0/1415803391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c146850 0x7f040c0715d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04080097a0 tx=0x7f040802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe76d28c5c23d4a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.614+0000 7f0411288640 1 -- 172.21.15.64:0/1415803391 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408031830 con 0x7f040c146850 2023-11-24T06:29:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.614+0000 7f0411288640 1 -- 172.21.15.64:0/1415803391 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0408031990 con 0x7f040c146850 2023-11-24T06:29:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.614+0000 7f0411288640 1 -- 172.21.15.64:0/1415803391 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408031d20 con 0x7f040c146850 2023-11-24T06:29:24.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 -- 172.21.15.64:0/1415803391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c146850 msgr2=0x7f040c0715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 --2- 172.21.15.64:0/1415803391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c146850 0x7f040c0715d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04080097a0 tx=0x7f040802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 -- 172.21.15.64:0/1415803391 shutdown_connections 2023-11-24T06:29:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 --2- 172.21.15.64:0/1415803391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c071b10 0x7f040c0ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 --2- 172.21.15.64:0/1415803391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c146850 0x7f040c0715d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 -- 172.21.15.64:0/1415803391 >> 172.21.15.64:0/1415803391 conn(0x7f040c06a3e0 msgr2=0x7f040c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 -- 172.21.15.64:0/1415803391 shutdown_connections 2023-11-24T06:29:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.615+0000 7f041328c640 1 -- 172.21.15.64:0/1415803391 wait complete. 2023-11-24T06:29:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.617+0000 7f041328c640 1 Processor -- start 2023-11-24T06:29:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.617+0000 7f041328c640 1 -- start start 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f041328c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c071b10 0x7f040c1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f041328c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c0adb00 0x7f040c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f041328c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c146850 0x7f040c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f041328c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c1f6140 con 0x7f040c146850 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f041328c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c1f6370 con 0x7f040c071b10 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f041328c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c1f6650 con 0x7f040c0adb00 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f0412a8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c146850 0x7f040c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f0412a8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c146850 0x7f040c0700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37362/0 (socket says 172.21.15.64:37362) 2023-11-24T06:29:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f0412a8b640 1 -- 172.21.15.64:0/1906415032 learned_addr learned my addr 172.21.15.64:0/1906415032 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f0411a89640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c0adb00 0x7f040c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.618+0000 7f041228a640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c071b10 0x7f040c1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0412a8b640 1 -- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c0adb00 msgr2=0x7f040c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0412a8b640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c0adb00 0x7f040c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0412a8b640 1 -- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c071b10 msgr2=0x7f040c1f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0412a8b640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c071b10 0x7f040c1f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0412a8b640 1 -- 172.21.15.64:0/1906415032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0408009400 con 0x7f040c146850 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f041228a640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c071b10 0x7f040c1f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0412a8b640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c146850 0x7f040c0700f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f040400cbc0 tx=0x7f040400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0402ffd640 1 -- 172.21.15.64:0/1906415032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040400ace0 con 0x7f040c146850 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0402ffd640 1 -- 172.21.15.64:0/1906415032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040400ae40 con 0x7f040c146850 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c1f6930 con 0x7f040c146850 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.619+0000 7f0402ffd640 1 -- 172.21.15.64:0/1906415032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404012600 con 0x7f040c146850 2023-11-24T06:29:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.620+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c15c9b0 con 0x7f040c146850 2023-11-24T06:29:24.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.621+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c0c41a0 con 0x7f040c146850 2023-11-24T06:29:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.622+0000 7f0402ffd640 1 -- 172.21.15.64:0/1906415032 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0404007070 con 0x7f040c146850 2023-11-24T06:29:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.623+0000 7f0402ffd640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03f006ae70 0x7f03f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.623+0000 7f0402ffd640 1 -- 172.21.15.64:0/1906415032 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04040893c0 con 0x7f040c146850 2023-11-24T06:29:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.623+0000 7f041228a640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03f006ae70 0x7f03f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.624+0000 7f041228a640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03f006ae70 0x7f03f006d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0408036000 tx=0x7f040803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:24.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.628+0000 7f0402ffd640 1 -- 172.21.15.64:0/1906415032 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f040c0c41a0 con 0x7f040c146850 2023-11-24T06:29:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.821+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f040c0c41a0 con 0x7f040c146850 2023-11-24T06:29:24.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.822+0000 7f0402ffd640 1 -- 172.21.15.64:0/1906415032 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f040c0c41a0 con 0x7f040c146850 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03f006ae70 msgr2=0x7f03f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03f006ae70 0x7f03f006d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0408036000 tx=0x7f040803c040 comp rx=0 tx=0).stop 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c146850 msgr2=0x7f040c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c146850 0x7f040c0700f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f040400cbc0 tx=0x7f040400a430 comp rx=0 tx=0).stop 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 shutdown_connections 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f03f006ae70 0x7f03f006d330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c146850 0x7f040c0700f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f040c0adb00 0x7f040c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 --2- 172.21.15.64:0/1906415032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f040c071b10 0x7f040c1f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:24.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 >> 172.21.15.64:0/1906415032 conn(0x7f040c06a3e0 msgr2=0x7f040c15fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:24.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 shutdown_connections 2023-11-24T06:29:24.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:24.825+0000 7f041328c640 1 -- 172.21.15.64:0/1906415032 wait complete. 2023-11-24T06:29:24.847 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:29:24.848 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:29:24.963 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:29:25.100 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:29:25.101 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:29:25.101 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:25.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:25.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.390+0000 7fbafdda0640 1 Processor -- start 2023-11-24T06:29:25.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.390+0000 7fbafdda0640 1 -- start start 2023-11-24T06:29:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafdda0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 0x7fbaf80d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafdda0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbaf80c6750 0x7fbaf80aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafdda0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf80c6be0 con 0x7fbaf815d590 2023-11-24T06:29:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafdda0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf80c7110 con 0x7fbaf80c6750 2023-11-24T06:29:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafdda0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80c6e30 con 0x7fbaf809a170 2023-11-24T06:29:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafd59f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbaf80c6750 0x7fbaf80aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafd59f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbaf80c6750 0x7fbaf80aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37370/0 (socket says 172.21.15.64:37370) 2023-11-24T06:29:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafd59f640 1 -- 172.21.15.64:0/3772225590 learned_addr learned my addr 172.21.15.64:0/3772225590 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:25.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafd59f640 1 -- 172.21.15.64:0/3772225590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 msgr2=0x7fbaf80d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:25.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafd59f640 1 --2- 172.21.15.64:0/3772225590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 0x7fbaf80d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.392+0000 7fbafd59f640 1 -- 172.21.15.64:0/3772225590 >> v1:172.21.15.64:6789/0 conn(0x7fbaf815d590 legacy=0x7fbaf8099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:25.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.393+0000 7fbafd59f640 1 -- 172.21.15.64:0/3772225590 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80adf30 con 0x7fbaf80c6750 2023-11-24T06:29:25.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.394+0000 7fbafd59f640 1 --2- 172.21.15.64:0/3772225590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf80c6750 0x7fbaf80aea10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009870 tx=0x7fbaf002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f595411861dc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.394+0000 7fbaf77fe640 1 -- 172.21.15.64:0/3772225590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0030840 con 0x7fbaf80c6750 2023-11-24T06:29:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.394+0000 7fbaf77fe640 1 -- 172.21.15.64:0/3772225590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf00309a0 con 0x7fbaf80c6750 2023-11-24T06:29:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.394+0000 7fbaf77fe640 1 -- 172.21.15.64:0/3772225590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0030d30 con 0x7fbaf80c6750 2023-11-24T06:29:25.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 -- 172.21.15.64:0/3772225590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf80c6750 msgr2=0x7fbaf80aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 --2- 172.21.15.64:0/3772225590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf80c6750 0x7fbaf80aea10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009870 tx=0x7fbaf002b980 comp rx=0 tx=0).stop 2023-11-24T06:29:25.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 -- 172.21.15.64:0/3772225590 shutdown_connections 2023-11-24T06:29:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 --2- 172.21.15.64:0/3772225590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf80c6750 0x7fbaf80aea10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 --2- 172.21.15.64:0/3772225590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 0x7fbaf80d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 -- 172.21.15.64:0/3772225590 >> 172.21.15.64:0/3772225590 conn(0x7fbaf80d0e80 msgr2=0x7fbaf80d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 -- 172.21.15.64:0/3772225590 shutdown_connections 2023-11-24T06:29:25.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.395+0000 7fbafdda0640 1 -- 172.21.15.64:0/3772225590 wait complete. 2023-11-24T06:29:25.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.397+0000 7fbafdda0640 1 Processor -- start 2023-11-24T06:29:25.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.397+0000 7fbafdda0640 1 -- start start 2023-11-24T06:29:25.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafdda0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 0x7fbaf814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafdda0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaf80c6750 0x7fbaf815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafdda0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf815d590 0x7fbaf815fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafdda0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf814f280 con 0x7fbaf815d590 2023-11-24T06:29:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafdda0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8160470 con 0x7fbaf80c6750 2023-11-24T06:29:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafdda0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8160750 con 0x7fbaf809a170 2023-11-24T06:29:25.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafd59f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf815d590 0x7fbaf815fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafd59f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf815d590 0x7fbaf815fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37374/0 (socket says 172.21.15.64:37374) 2023-11-24T06:29:25.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.399+0000 7fbafd59f640 1 -- 172.21.15.64:0/467404580 learned_addr learned my addr 172.21.15.64:0/467404580 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:25.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbafcd9e640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 0x7fbaf814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbafd59f640 1 -- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 msgr2=0x7fbaf814ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbafd59f640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 0x7fbaf814ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbaf7fff640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaf80c6750 0x7fbaf815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbafd59f640 1 -- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaf80c6750 msgr2=0x7fbaf815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbafd59f640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaf80c6750 0x7fbaf815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbafd59f640 1 -- 172.21.15.64:0/467404580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00094d0 con 0x7fbaf815d590 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbaf7fff640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaf80c6750 0x7fbaf815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbafd59f640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf815d590 0x7fbaf815fdf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002b9c0 tx=0x7fbaf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.400+0000 7fbaf57fa640 1 -- 172.21.15.64:0/467404580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0039e20 con 0x7fbaf815d590 2023-11-24T06:29:25.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.401+0000 7fbaf57fa640 1 -- 172.21.15.64:0/467404580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf0043400 con 0x7fbaf815d590 2023-11-24T06:29:25.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.401+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf8160a30 con 0x7fbaf815d590 2023-11-24T06:29:25.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.401+0000 7fbaf57fa640 1 -- 172.21.15.64:0/467404580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00437b0 con 0x7fbaf815d590 2023-11-24T06:29:25.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.401+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf80b5940 con 0x7fbaf815d590 2023-11-24T06:29:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.402+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8160c70 con 0x7fbaf815d590 2023-11-24T06:29:25.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.403+0000 7fbaf57fa640 1 -- 172.21.15.64:0/467404580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbaf0043910 con 0x7fbaf815d590 2023-11-24T06:29:25.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.404+0000 7fbaf57fa640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbae406ae40 0x7fbae406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.404+0000 7fbaf57fa640 1 -- 172.21.15.64:0/467404580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbaf003e070 con 0x7fbaf815d590 2023-11-24T06:29:25.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.404+0000 7fbafcd9e640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbae406ae40 0x7fbae406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.405+0000 7fbafcd9e640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbae406ae40 0x7fbae406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbae80012e0 tx=0x7fbae8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.409+0000 7fbaf57fa640 1 -- 172.21.15.64:0/467404580 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbaf8160c70 con 0x7fbaf815d590 2023-11-24T06:29:25.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.606+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbaf80cdce0 con 0x7fbaf815d590 2023-11-24T06:29:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.609+0000 7fbaf57fa640 1 -- 172.21.15.64:0/467404580 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fbaf80cdce0 con 0x7fbaf815d590 2023-11-24T06:29:25.612 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:12.271627+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:25.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.611+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbae406ae40 msgr2=0x7fbae406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.611+0000 7fbafdda0640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbae406ae40 0x7fbae406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbae80012e0 tx=0x7fbae8009040 comp rx=0 tx=0).stop 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.611+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf815d590 msgr2=0x7fbaf815fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.611+0000 7fbafdda0640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf815d590 0x7fbaf815fdf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002b9c0 tx=0x7fbaf0004730 comp rx=0 tx=0).stop 2023-11-24T06:29:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 shutdown_connections 2023-11-24T06:29:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fbae406ae40 0x7fbae406d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbaf815d590 0x7fbaf815fdf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaf80c6750 0x7fbaf815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 --2- 172.21.15.64:0/467404580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbaf809a170 0x7fbaf814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 >> 172.21.15.64:0/467404580 conn(0x7fbaf80d0e80 msgr2=0x7fbaf815bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 shutdown_connections 2023-11-24T06:29:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.612+0000 7fbafdda0640 1 -- 172.21.15.64:0/467404580 wait complete. 2023-11-24T06:29:25.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:08.540135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:09.752538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:25.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.863+0000 7f12f3abb640 1 Processor -- start 2023-11-24T06:29:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.864+0000 7f12f3abb640 1 -- start start 2023-11-24T06:29:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.865+0000 7f12f3abb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec13df80 0x7f12ec15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.866+0000 7f12f3abb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec06c470 con 0x7f12ec0cf9a0 2023-11-24T06:29:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.866+0000 7f12f3abb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec06c750 con 0x7f12ec13e360 2023-11-24T06:29:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.866+0000 7f12f3abb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec06ca30 con 0x7f12ec13df80 2023-11-24T06:29:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.866+0000 7f12f2058640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f12ec0cf9a0 0x7f12ec0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57600/0 (socket says 172.21.15.64:57600) 2023-11-24T06:29:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.866+0000 7f12f2058640 1 -- 172.21.15.64:0/2344246520 learned_addr learned my addr 172.21.15.64:0/2344246520 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.866+0000 7f12f1857640 1 --2- 172.21.15.64:0/2344246520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec13df80 0x7f12ec15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.867+0000 7f12f0855640 1 -- 172.21.15.64:0/2344246520 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283495866 0 0) 0x7f12ec06c470 con 0x7f12ec0cf9a0 2023-11-24T06:29:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.867+0000 7f12f0855640 1 -- 172.21.15.64:0/2344246520 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d8003520 con 0x7f12ec0cf9a0 2023-11-24T06:29:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.867+0000 7f12f1857640 1 -- 172.21.15.64:0/2344246520 >> v1:172.21.15.162:6789/0 conn(0x7f12ec13e360 legacy=0x7f12ec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.867+0000 7f12f1857640 1 -- 172.21.15.64:0/2344246520 >> v1:172.21.15.64:6789/0 conn(0x7f12ec0cf9a0 legacy=0x7f12ec0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.867+0000 7f12f1857640 1 -- 172.21.15.64:0/2344246520 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec06cd10 con 0x7f12ec13df80 2023-11-24T06:29:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.869+0000 7f12f1857640 1 --2- 172.21.15.64:0/2344246520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec13df80 0x7f12ec15bee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12e80097d0 tx=0x7f12e802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef266a21c9a5810 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.869+0000 7f12f0855640 1 -- 172.21.15.64:0/2344246520 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8030800 con 0x7f12ec13df80 2023-11-24T06:29:25.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.869+0000 7f12f0855640 1 -- 172.21.15.64:0/2344246520 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e8030960 con 0x7f12ec13df80 2023-11-24T06:29:25.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.869+0000 7f12f0855640 1 -- 172.21.15.64:0/2344246520 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8030cf0 con 0x7f12ec13df80 2023-11-24T06:29:25.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.870+0000 7f12f3abb640 1 -- 172.21.15.64:0/2344246520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec13df80 msgr2=0x7f12ec15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.870+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2344246520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec13df80 0x7f12ec15bee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12e80097d0 tx=0x7f12e802b940 comp rx=0 tx=0).stop 2023-11-24T06:29:25.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.870+0000 7f12f3abb640 1 -- 172.21.15.64:0/2344246520 shutdown_connections 2023-11-24T06:29:25.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.870+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2344246520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec13df80 0x7f12ec15bee0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.870+0000 7f12f3abb640 1 -- 172.21.15.64:0/2344246520 >> 172.21.15.64:0/2344246520 conn(0x7f12ec06a800 msgr2=0x7f12ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:25.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.870+0000 7f12f3abb640 1 -- 172.21.15.64:0/2344246520 shutdown_connections 2023-11-24T06:29:25.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.870+0000 7f12f3abb640 1 -- 172.21.15.64:0/2344246520 wait complete. 2023-11-24T06:29:25.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.872+0000 7f12f3abb640 1 Processor -- start 2023-11-24T06:29:25.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.872+0000 7f12f3abb640 1 -- start start 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.873+0000 7f12f3abb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec0cf9a0 0x7f12ec071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.873+0000 7f12f3abb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12ec13df80 0x7f12ec1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.873+0000 7f12f3abb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec13e360 0x7f12ec1f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.873+0000 7f12f3abb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec153ea0 con 0x7f12ec13e360 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.873+0000 7f12f3abb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec153290 con 0x7f12ec13df80 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.873+0000 7f12f3abb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec153570 con 0x7f12ec0cf9a0 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec13e360 0x7f12ec1f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec13e360 0x7f12ec1f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37390/0 (socket says 172.21.15.64:37390) 2023-11-24T06:29:25.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 -- 172.21.15.64:0/2031131122 learned_addr learned my addr 172.21.15.64:0/2031131122 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:25.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f1857640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec0cf9a0 0x7f12ec071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f1056640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12ec13df80 0x7f12ec1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 -- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec0cf9a0 msgr2=0x7f12ec071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec0cf9a0 0x7f12ec071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 -- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12ec13df80 msgr2=0x7f12ec1f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12ec13df80 0x7f12ec1f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 -- 172.21.15.64:0/2031131122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8009400 con 0x7f12ec13e360 2023-11-24T06:29:25.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.874+0000 7f12f2058640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec13e360 0x7f12ec1f6140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12e400caa0 tx=0x7f12e400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.875+0000 7f12e27fc640 1 -- 172.21.15.64:0/2031131122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400ac00 con 0x7f12ec13e360 2023-11-24T06:29:25.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.875+0000 7f12e27fc640 1 -- 172.21.15.64:0/2031131122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400ad60 con 0x7f12ec13e360 2023-11-24T06:29:25.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.875+0000 7f12e27fc640 1 -- 172.21.15.64:0/2031131122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e4012620 con 0x7f12ec13e360 2023-11-24T06:29:25.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.875+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec153850 con 0x7f12ec13e360 2023-11-24T06:29:25.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.875+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec071940 con 0x7f12ec13e360 2023-11-24T06:29:25.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.876+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec071b60 con 0x7f12ec13e360 2023-11-24T06:29:25.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.877+0000 7f12e27fc640 1 -- 172.21.15.64:0/2031131122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f12e4007070 con 0x7f12ec13e360 2023-11-24T06:29:25.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.878+0000 7f12e27fc640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f12d006ae40 0x7f12d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.878+0000 7f12e27fc640 1 -- 172.21.15.64:0/2031131122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12e4089530 con 0x7f12ec13e360 2023-11-24T06:29:25.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.878+0000 7f12f1857640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f12d006ae40 0x7f12d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.879+0000 7f12f1857640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f12d006ae40 0x7f12d006d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12e8036000 tx=0x7f12e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:25.883+0000 7f12e27fc640 1 -- 172.21.15.64:0/2031131122 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12e4054670 con 0x7f12ec13e360 2023-11-24T06:29:26.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.077+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12ec0675d0 con 0x7f12ec13e360 2023-11-24T06:29:26.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.078+0000 7f12e27fc640 1 -- 172.21.15.64:0/2031131122 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12e4057c90 con 0x7f12ec13e360 2023-11-24T06:29:26.081 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:29:26.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f12d006ae40 msgr2=0x7f12d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f12d006ae40 0x7f12d006d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12e8036000 tx=0x7f12e8008030 comp rx=0 tx=0).stop 2023-11-24T06:29:26.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec13e360 msgr2=0x7f12ec1f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec13e360 0x7f12ec1f6140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12e400caa0 tx=0x7f12e400cf60 comp rx=0 tx=0).stop 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 shutdown_connections 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f12d006ae40 0x7f12d006d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec13e360 0x7f12ec1f6140 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12ec13df80 0x7f12ec1f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 --2- 172.21.15.64:0/2031131122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12ec0cf9a0 0x7f12ec071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 >> 172.21.15.64:0/2031131122 conn(0x7f12ec06a800 msgr2=0x7f12ec0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 shutdown_connections 2023-11-24T06:29:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.081+0000 7f12f3abb640 1 -- 172.21.15.64:0/2031131122 wait complete. 2023-11-24T06:29:26.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:29:26.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.330+0000 7f758de1a640 1 Processor -- start 2023-11-24T06:29:26.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.330+0000 7f758de1a640 1 -- start start 2023-11-24T06:29:26.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.332+0000 7f758de1a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f75880ccb50 0x7f7588070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.332+0000 7f758de1a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758815f4e0 con 0x7f75880adf20 2023-11-24T06:29:26.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.332+0000 7f758de1a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7588071130 con 0x7f758815bf10 2023-11-24T06:29:26.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.332+0000 7f758de1a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758815f970 con 0x7f75880ccb50 2023-11-24T06:29:26.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.332+0000 7f758ce18640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f75880adf20 0x7f75880cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57606/0 (socket says 172.21.15.64:57606) 2023-11-24T06:29:26.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.332+0000 7f758ce18640 1 -- 172.21.15.64:0/2583569110 learned_addr learned my addr 172.21.15.64:0/2583569110 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:26.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824136173 0 0) 0x7f758815f4e0 con 0x7f75880adf20 2023-11-24T06:29:26.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7578003520 con 0x7f75880adf20 2023-11-24T06:29:26.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f7587fff640 1 --2- 172.21.15.64:0/2583569110 >> v2:172.21.15.162:3300/0 conn(0x7f75880ccb50 0x7f7588070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041810887 0 0) 0x7f7578003520 con 0x7f75880adf20 2023-11-24T06:29:26.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 >> v2:172.21.15.162:3300/0 conn(0x7f75880ccb50 msgr2=0x7f7588070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 --2- 172.21.15.64:0/2583569110 >> v2:172.21.15.162:3300/0 conn(0x7f75880ccb50 0x7f7588070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 >> v1:172.21.15.162:6789/0 conn(0x7f758815bf10 legacy=0x7f7588143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758815ebb0 con 0x7f75880adf20 2023-11-24T06:29:26.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f757c001670 con 0x7f75880adf20 2023-11-24T06:29:26.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 >> v1:172.21.15.64:6789/0 conn(0x7f75880adf20 legacy=0x7f75880cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7578003970 0x7f7578003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f7578004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75780051d0 0x7f757800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758815f970 con 0x7f75780047c0 2023-11-24T06:29:26.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757800c100 con 0x7f7578003970 2023-11-24T06:29:26.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757800c2c0 con 0x7f75780051d0 2023-11-24T06:29:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.333+0000 7f7587fff640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f7578004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f758ce18640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7578003970 0x7f7578003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f758d619640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75780051d0 0x7f757800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f7587fff640 1 -- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75780051d0 msgr2=0x7f757800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f7587fff640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75780051d0 0x7f757800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f7587fff640 1 -- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7578003970 msgr2=0x7f7578003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f7587fff640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7578003970 0x7f7578003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f7587fff640 1 -- 172.21.15.64:0/2583569110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7578003520 con 0x7f75780047c0 2023-11-24T06:29:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.334+0000 7f758d619640 1 -- 172.21.15.64:0/2583569110 reap_dead start 2023-11-24T06:29:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.335+0000 7f7587fff640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f7578004c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f75880ccf90 tx=0x7f75780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a1e10a64f52850 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574033670 con 0x7f75780047c0 2023-11-24T06:29:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75740337d0 con 0x7f75780047c0 2023-11-24T06:29:26.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f75877fe640 1 -- 172.21.15.64:0/2583569110 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574033b60 con 0x7f75780047c0 2023-11-24T06:29:26.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f758de1a640 1 -- 172.21.15.64:0/2583569110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 msgr2=0x7f7578004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f758de1a640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f7578004c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f75880ccf90 tx=0x7f75780057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f758de1a640 1 -- 172.21.15.64:0/2583569110 shutdown_connections 2023-11-24T06:29:26.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f758de1a640 1 --2- 172.21.15.64:0/2583569110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f7578004c40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f758de1a640 1 -- 172.21.15.64:0/2583569110 >> 172.21.15.64:0/2583569110 conn(0x7f758806a7d0 msgr2=0x7f758806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:26.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f758de1a640 1 -- 172.21.15.64:0/2583569110 shutdown_connections 2023-11-24T06:29:26.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.336+0000 7f758de1a640 1 -- 172.21.15.64:0/2583569110 wait complete. 2023-11-24T06:29:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.338+0000 7f758de1a640 1 Processor -- start 2023-11-24T06:29:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.338+0000 7f758de1a640 1 -- start start 2023-11-24T06:29:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.339+0000 7f758de1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f758815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.339+0000 7f758de1a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7588137190 0x7f75880ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.339+0000 7f758de1a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75880b3480 0x7f75880b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.339+0000 7f758de1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880b3e40 con 0x7f75780047c0 2023-11-24T06:29:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.339+0000 7f758de1a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880caef0 con 0x7f7588137190 2023-11-24T06:29:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.339+0000 7f758de1a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75880c81e0 con 0x7f75880b3480 2023-11-24T06:29:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.339+0000 7f758ce18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f758815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f758815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37414/0 (socket says 172.21.15.64:37414) 2023-11-24T06:29:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 -- 172.21.15.64:0/2720964405 learned_addr learned my addr 172.21.15.64:0/2720964405 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f7587fff640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7588137190 0x7f75880ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758d619640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75880b3480 0x7f75880b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 -- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75880b3480 msgr2=0x7f75880b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75880b3480 0x7f75880b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 -- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7588137190 msgr2=0x7f75880ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7588137190 0x7f75880ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 -- 172.21.15.64:0/2720964405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588135140 con 0x7f75780047c0 2023-11-24T06:29:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f7587fff640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7588137190 0x7f75880ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.340+0000 7f758ce18640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f758815aaa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e790 tx=0x7f757c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.341+0000 7f75857fa640 1 -- 172.21.15.64:0/2720964405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00eea0 con 0x7f75780047c0 2023-11-24T06:29:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.341+0000 7f75857fa640 1 -- 172.21.15.64:0/2720964405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c00c870 con 0x7f75780047c0 2023-11-24T06:29:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.341+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7588137740 con 0x7f75780047c0 2023-11-24T06:29:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.341+0000 7f75857fa640 1 -- 172.21.15.64:0/2720964405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00cc00 con 0x7f75780047c0 2023-11-24T06:29:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.341+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588137d00 con 0x7f75780047c0 2023-11-24T06:29:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.342+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7588067450 con 0x7f75780047c0 2023-11-24T06:29:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.343+0000 7f75857fa640 1 -- 172.21.15.64:0/2720964405 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f757c00cd60 con 0x7f75780047c0 2023-11-24T06:29:26.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.344+0000 7f75857fa640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f756c06ae40 0x7f756c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.344+0000 7f75857fa640 1 -- 172.21.15.64:0/2720964405 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f757c089500 con 0x7f75780047c0 2023-11-24T06:29:26.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.344+0000 7f7587fff640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f756c06ae40 0x7f756c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.345+0000 7f7587fff640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f756c06ae40 0x7f756c06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f75780057a0 tx=0x7f7574030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.349+0000 7f75857fa640 1 -- 172.21.15.64:0/2720964405 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f757c0545a0 con 0x7f75780047c0 2023-11-24T06:29:26.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.543+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75880b0c10 con 0x7f75780047c0 2023-11-24T06:29:26.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.544+0000 7f75857fa640 1 -- 172.21.15.64:0/2720964405 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f757c057bc0 con 0x7f75780047c0 2023-11-24T06:29:26.547 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:29:26.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f756c06ae40 msgr2=0x7f756c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f756c06ae40 0x7f756c06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f75780057a0 tx=0x7f7574030000 comp rx=0 tx=0).stop 2023-11-24T06:29:26.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 msgr2=0x7f758815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f758815aaa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e790 tx=0x7f757c00ec50 comp rx=0 tx=0).stop 2023-11-24T06:29:26.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 shutdown_connections 2023-11-24T06:29:26.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f756c06ae40 0x7f756c06d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75880b3480 0x7f75880b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7588137190 0x7f75880ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 --2- 172.21.15.64:0/2720964405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75780047c0 0x7f758815aaa0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 >> 172.21.15.64:0/2720964405 conn(0x7f758806a7d0 msgr2=0x7f75880cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:26.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.547+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 shutdown_connections 2023-11-24T06:29:26.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.548+0000 7f758de1a640 1 -- 172.21.15.64:0/2720964405 wait complete. 2023-11-24T06:29:26.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:29:26.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.795+0000 7fced8f1d640 1 Processor -- start 2023-11-24T06:29:26.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.795+0000 7fced8f1d640 1 -- start start 2023-11-24T06:29:26.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.797+0000 7fced8f1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40d0b60 0x7fced40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.797+0000 7fced8f1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fced40aede0 0x7fced406f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.797+0000 7fced8f1d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced4071de0 con 0x7fced406a3e0 2023-11-24T06:29:26.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.797+0000 7fced8f1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced40af270 con 0x7fced40aede0 2023-11-24T06:29:26.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.797+0000 7fced8f1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced4072030 con 0x7fced40d0b60 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.797+0000 7fced2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fced40aede0 0x7fced406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.797+0000 7fced2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fced40aede0 0x7fced406f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37418/0 (socket says 172.21.15.64:37418) 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.798+0000 7fced2d9d640 1 -- 172.21.15.64:0/700692983 learned_addr learned my addr 172.21.15.64:0/700692983 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.798+0000 7fcec9d9b640 1 --2- 172.21.15.64:0/700692983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40d0b60 0x7fced40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.798+0000 7fced2d9d640 1 -- 172.21.15.64:0/700692983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40d0b60 msgr2=0x7fced40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.798+0000 7fced2d9d640 1 --2- 172.21.15.64:0/700692983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40d0b60 0x7fced40ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.798+0000 7fced2d9d640 1 -- 172.21.15.64:0/700692983 >> v1:172.21.15.162:6789/0 conn(0x7fced406a3e0 legacy=0x7fced406a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.798+0000 7fced2d9d640 1 -- 172.21.15.64:0/700692983 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4072310 con 0x7fced40aede0 2023-11-24T06:29:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.799+0000 7fced2d9d640 1 --2- 172.21.15.64:0/700692983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced40aede0 0x7fced406f080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009e50 tx=0x7fcecc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c68790cec2b8da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.799+0000 7fced1d9b640 1 -- 172.21.15.64:0/700692983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc02ca70 con 0x7fced40aede0 2023-11-24T06:29:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced1d9b640 1 -- 172.21.15.64:0/700692983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc02cbd0 con 0x7fced40aede0 2023-11-24T06:29:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced1d9b640 1 -- 172.21.15.64:0/700692983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0343f0 con 0x7fced40aede0 2023-11-24T06:29:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 -- 172.21.15.64:0/700692983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced40aede0 msgr2=0x7fced406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 --2- 172.21.15.64:0/700692983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced40aede0 0x7fced406f080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009e50 tx=0x7fcecc02bed0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 -- 172.21.15.64:0/700692983 shutdown_connections 2023-11-24T06:29:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 --2- 172.21.15.64:0/700692983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced40aede0 0x7fced406f080 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 --2- 172.21.15.64:0/700692983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40d0b60 0x7fced40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 -- 172.21.15.64:0/700692983 >> 172.21.15.64:0/700692983 conn(0x7fced40cfef0 msgr2=0x7fced40d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:26.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 -- 172.21.15.64:0/700692983 shutdown_connections 2023-11-24T06:29:26.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.800+0000 7fced8f1d640 1 -- 172.21.15.64:0/700692983 wait complete. 2023-11-24T06:29:26.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.802+0000 7fced8f1d640 1 Processor -- start 2023-11-24T06:29:26.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.802+0000 7fced8f1d640 1 -- start start 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.803+0000 7fced8f1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced406a3e0 0x7fced415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.803+0000 7fced8f1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40aede0 0x7fced415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.803+0000 7fced8f1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fced40d0b60 0x7fced41f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.803+0000 7fced8f1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced4145390 con 0x7fced406a3e0 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.803+0000 7fced8f1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced415b1f0 con 0x7fced40d0b60 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.803+0000 7fced8f1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced415b4d0 con 0x7fced40aede0 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.803+0000 7fced259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced406a3e0 0x7fced415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced406a3e0 0x7fced415a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37422/0 (socket says 172.21.15.64:37422) 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 -- 172.21.15.64:0/3582654060 learned_addr learned my addr 172.21.15.64:0/3582654060 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fcec9d9b640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40aede0 0x7fced415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced2d9d640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fced40d0b60 0x7fced41f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 -- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40aede0 msgr2=0x7fced415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40aede0 0x7fced415a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 -- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fced40d0b60 msgr2=0x7fced41f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fced40d0b60 0x7fced41f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 -- 172.21.15.64:0/3582654060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc009b30 con 0x7fced406a3e0 2023-11-24T06:29:26.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.804+0000 7fced259c640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced406a3e0 0x7fced415a480 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c430 tx=0x7fcec400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.805+0000 7fcecb7fe640 1 -- 172.21.15.64:0/3582654060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4011790 con 0x7fced406a3e0 2023-11-24T06:29:26.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.805+0000 7fcecb7fe640 1 -- 172.21.15.64:0/3582654060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec40118f0 con 0x7fced406a3e0 2023-11-24T06:29:26.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.805+0000 7fcecb7fe640 1 -- 172.21.15.64:0/3582654060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4011ce0 con 0x7fced406a3e0 2023-11-24T06:29:26.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.805+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced415b7b0 con 0x7fced406a3e0 2023-11-24T06:29:26.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.805+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced415af00 con 0x7fced406a3e0 2023-11-24T06:29:26.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.806+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced40675d0 con 0x7fced406a3e0 2023-11-24T06:29:26.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.807+0000 7fcecb7fe640 1 -- 172.21.15.64:0/3582654060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcec401c460 con 0x7fced406a3e0 2023-11-24T06:29:26.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.808+0000 7fcecb7fe640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fceb406ae40 0x7fceb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.809+0000 7fcecb7fe640 1 -- 172.21.15.64:0/3582654060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcec40895f0 con 0x7fced406a3e0 2023-11-24T06:29:26.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.809+0000 7fcec9d9b640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fceb406ae40 0x7fceb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.810+0000 7fcec9d9b640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fceb406ae40 0x7fceb406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcebc006fd0 tx=0x7fcebc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:26.813+0000 7fcecb7fe640 1 -- 172.21.15.64:0/3582654060 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcec4054680 con 0x7fced406a3e0 2023-11-24T06:29:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.009+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fced406a8c0 con 0x7fced406a3e0 2023-11-24T06:29:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.010+0000 7fcecb7fe640 1 -- 172.21.15.64:0/3582654060 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcec4057ca0 con 0x7fced406a3e0 2023-11-24T06:29:27.013 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:29:27.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fceb406ae40 msgr2=0x7fceb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fceb406ae40 0x7fceb406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcebc006fd0 tx=0x7fcebc008040 comp rx=0 tx=0).stop 2023-11-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced406a3e0 msgr2=0x7fced415a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced406a3e0 0x7fced415a480 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c430 tx=0x7fcec400c8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 shutdown_connections 2023-11-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fceb406ae40 0x7fceb406d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fced40d0b60 0x7fced41f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fced40aede0 0x7fced415a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.012+0000 7fced8f1d640 1 --2- 172.21.15.64:0/3582654060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fced406a3e0 0x7fced415a480 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.013+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 >> 172.21.15.64:0/3582654060 conn(0x7fced40cfef0 msgr2=0x7fced40acec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.013+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 shutdown_connections 2023-11-24T06:29:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.013+0000 7fced8f1d640 1 -- 172.21.15.64:0/3582654060 wait complete. 2023-11-24T06:29:27.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:29:27.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.247+0000 7fcac0967640 1 Processor -- start 2023-11-24T06:29:27.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.248+0000 7fcac0967640 1 -- start start 2023-11-24T06:29:27.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.249+0000 7fcac0967640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcabc160920 0x7fcabc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.249+0000 7fcac0967640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc0d0a10 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.249+0000 7fcac0967640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc161e30 con 0x7fcabc0cfea0 2023-11-24T06:29:27.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.249+0000 7fcac0967640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcabc0d1f50 con 0x7fcabc160920 2023-11-24T06:29:27.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.249+0000 7fcabad9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcabc160920 0x7fcabc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.249+0000 7fcabad9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcabc160920 0x7fcabc0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37432/0 (socket says 172.21.15.64:37432) 2023-11-24T06:29:27.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.249+0000 7fcabad9d640 1 -- 172.21.15.64:0/986226730 learned_addr learned my addr 172.21.15.64:0/986226730 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:27.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.250+0000 7fcabad9d640 1 -- 172.21.15.64:0/986226730 >> v1:172.21.15.162:6789/0 conn(0x7fcabc0cfea0 legacy=0x7fcabc161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:27.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.250+0000 7fcabad9d640 1 -- 172.21.15.64:0/986226730 >> v1:172.21.15.64:6789/0 conn(0x7fcabc0cf9f0 legacy=0x7fcabc161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.250+0000 7fcabad9d640 1 -- 172.21.15.64:0/986226730 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0cf080 con 0x7fcabc160920 2023-11-24T06:29:27.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.251+0000 7fcabad9d640 1 --2- 172.21.15.64:0/986226730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc160920 0x7fcabc0b3f30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d184b2bd77d60aae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.251+0000 7fcab9d9b640 1 -- 172.21.15.64:0/986226730 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac030860 con 0x7fcabc160920 2023-11-24T06:29:27.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.251+0000 7fcab9d9b640 1 -- 172.21.15.64:0/986226730 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac0309c0 con 0x7fcabc160920 2023-11-24T06:29:27.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.251+0000 7fcab9d9b640 1 -- 172.21.15.64:0/986226730 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac030d50 con 0x7fcabc160920 2023-11-24T06:29:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.252+0000 7fcac0967640 1 -- 172.21.15.64:0/986226730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc160920 msgr2=0x7fcabc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.252+0000 7fcac0967640 1 --2- 172.21.15.64:0/986226730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc160920 0x7fcabc0b3f30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.252+0000 7fcac0967640 1 -- 172.21.15.64:0/986226730 shutdown_connections 2023-11-24T06:29:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.252+0000 7fcac0967640 1 --2- 172.21.15.64:0/986226730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc160920 0x7fcabc0b3f30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.252+0000 7fcac0967640 1 -- 172.21.15.64:0/986226730 >> 172.21.15.64:0/986226730 conn(0x7fcabc0d3090 msgr2=0x7fcabc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.252+0000 7fcac0967640 1 -- 172.21.15.64:0/986226730 shutdown_connections 2023-11-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.252+0000 7fcac0967640 1 -- 172.21.15.64:0/986226730 wait complete. 2023-11-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.254+0000 7fcac0967640 1 Processor -- start 2023-11-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.254+0000 7fcac0967640 1 -- start start 2023-11-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcac0967640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc0cf9f0 0x7fcabc0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcac0967640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcabc0cfea0 0x7fcabc150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcac0967640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcabc160920 0x7fcabc0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcac0967640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0ccca0 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcac0967640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc153fd0 con 0x7fcabc160920 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcac0967640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc1518f0 con 0x7fcabc0cfea0 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcabad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc0cf9f0 0x7fcabc0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcabad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc0cf9f0 0x7fcabc0cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37440/0 (socket says 172.21.15.64:37440) 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.255+0000 7fcabad9d640 1 -- 172.21.15.64:0/2311690862 learned_addr learned my addr 172.21.15.64:0/2311690862 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcabad9d640 1 -- 172.21.15.64:0/2311690862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcabc0cfea0 msgr2=0x7fcabc150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcabad9d640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcabc0cfea0 0x7fcabc150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcabad9d640 1 -- 172.21.15.64:0/2311690862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcabc160920 msgr2=0x7fcabc0c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcabad9d640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcabc160920 0x7fcabc0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcabad9d640 1 -- 172.21.15.64:0/2311690862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0094d0 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcabad9d640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc0cf9f0 0x7fcabc0cab70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcab37fe640 1 -- 172.21.15.64:0/2311690862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac030810 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcab37fe640 1 -- 172.21.15.64:0/2311690862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac043400 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcab37fe640 1 -- 172.21.15.64:0/2311690862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac0437b0 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.256+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc0cc090 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.257+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc0cc650 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.258+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc0cce10 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.258+0000 7fcab37fe640 1 -- 172.21.15.64:0/2311690862 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaac043910 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.259+0000 7fcab37fe640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fca9c06ae70 0x7fca9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.259+0000 7fcab37fe640 1 -- 172.21.15.64:0/2311690862 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaac03e070 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.260+0000 7fcaba59c640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fca9c06ae70 0x7fca9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.260+0000 7fcaba59c640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fca9c06ae70 0x7fca9c06d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40012e0 tx=0x7fcaa4007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.264+0000 7fcab37fe640 1 -- 172.21.15.64:0/2311690862 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaac077ae0 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.475+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcabc067450 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.476+0000 7fcab37fe640 1 -- 172.21.15.64:0/2311690862 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcaac07b100 con 0x7fcabc0cf9f0 2023-11-24T06:29:27.479 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:29:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fca9c06ae70 msgr2=0x7fca9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fca9c06ae70 0x7fca9c06d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40012e0 tx=0x7fcaa4007d30 comp rx=0 tx=0).stop 2023-11-24T06:29:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc0cf9f0 msgr2=0x7fcabc0cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc0cf9f0 0x7fcabc0cab70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac004730 comp rx=0 tx=0).stop 2023-11-24T06:29:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 shutdown_connections 2023-11-24T06:29:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fca9c06ae70 0x7fca9c06d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcabc160920 0x7fcabc0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcabc0cfea0 0x7fcabc150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 --2- 172.21.15.64:0/2311690862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcabc0cf9f0 0x7fcabc0cab70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.478+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 >> 172.21.15.64:0/2311690862 conn(0x7fcabc0d3090 msgr2=0x7fcabc15b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.479+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 shutdown_connections 2023-11-24T06:29:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.479+0000 7fcac0967640 1 -- 172.21.15.64:0/2311690862 wait complete. 2023-11-24T06:29:27.501 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T06:29:27.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:27.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:27.589 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[101834]: fuse finished with error 0 and tester_r 0 2023-11-24T06:29:27.612 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:29:27.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:27.612 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:29:27.671 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:29:27.671 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:27.732 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:27.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.964+0000 7f688ffff640 1 Processor -- start 2023-11-24T06:29:27.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.964+0000 7f688ffff640 1 -- start start 2023-11-24T06:29:27.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6890143c40 0x7f6890071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6890070e50 con 0x7f68900717d0 2023-11-24T06:29:27.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688ffff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6890144020 con 0x7f68900ccc10 2023-11-24T06:29:27.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900723c0 con 0x7f6890143c40 2023-11-24T06:29:27.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6890143c40 0x7f6890071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6890143c40 0x7f6890071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37454/0 (socket says 172.21.15.64:37454) 2023-11-24T06:29:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688effd640 1 -- 172.21.15.64:0/3091047532 learned_addr learned my addr 172.21.15.64:0/3091047532 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.966+0000 7f688e7fc640 1 --1- 172.21.15.64:0/3091047532 >> v1:172.21.15.64:6789/0 conn(0x7f68900717d0 0x7f68900cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57610/0 (socket says 172.21.15.64:57610) 2023-11-24T06:29:27.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.967+0000 7f688effd640 1 -- 172.21.15.64:0/3091047532 >> v1:172.21.15.162:6789/0 conn(0x7f68900ccc10 legacy=0x7f689006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:27.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.967+0000 7f688effd640 1 -- 172.21.15.64:0/3091047532 >> v1:172.21.15.64:6789/0 conn(0x7f68900717d0 legacy=0x7f68900cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.967+0000 7f688effd640 1 -- 172.21.15.64:0/3091047532 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890071e00 con 0x7f6890143c40 2023-11-24T06:29:27.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.968+0000 7f688effd640 1 --2- 172.21.15.64:0/3091047532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6890143c40 0x7f6890071290 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6880009870 tx=0x7f688002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c9a4cc43a0c251 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.968+0000 7f688dffb640 1 -- 172.21.15.64:0/3091047532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880031870 con 0x7f6890143c40 2023-11-24T06:29:27.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.968+0000 7f688dffb640 1 -- 172.21.15.64:0/3091047532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68800319d0 con 0x7f6890143c40 2023-11-24T06:29:27.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.968+0000 7f688dffb640 1 -- 172.21.15.64:0/3091047532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880031d60 con 0x7f6890143c40 2023-11-24T06:29:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.969+0000 7f688ffff640 1 -- 172.21.15.64:0/3091047532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6890143c40 msgr2=0x7f6890071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.969+0000 7f688ffff640 1 --2- 172.21.15.64:0/3091047532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6890143c40 0x7f6890071290 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6880009870 tx=0x7f688002b910 comp rx=0 tx=0).stop 2023-11-24T06:29:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.969+0000 7f688ffff640 1 -- 172.21.15.64:0/3091047532 shutdown_connections 2023-11-24T06:29:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.969+0000 7f688ffff640 1 --2- 172.21.15.64:0/3091047532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6890143c40 0x7f6890071290 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.969+0000 7f688ffff640 1 -- 172.21.15.64:0/3091047532 >> 172.21.15.64:0/3091047532 conn(0x7f689006a800 msgr2=0x7f689006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.969+0000 7f688ffff640 1 -- 172.21.15.64:0/3091047532 shutdown_connections 2023-11-24T06:29:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.969+0000 7f688ffff640 1 -- 172.21.15.64:0/3091047532 wait complete. 2023-11-24T06:29:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.971+0000 7f688ffff640 1 Processor -- start 2023-11-24T06:29:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.971+0000 7f688ffff640 1 -- start start 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68900717d0 0x7f689015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68900ccc10 0x7f68900c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6890143c40 0x7f68900c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689015aec0 con 0x7f68900ccc10 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688ffff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900c4c30 con 0x7f68900717d0 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688ffff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900c4f10 con 0x7f6890143c40 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68900ccc10 0x7f68900c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68900ccc10 0x7f68900c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37456/0 (socket says 172.21.15.64:37456) 2023-11-24T06:29:27.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.972+0000 7f688e7fc640 1 -- 172.21.15.64:0/6318858 learned_addr learned my addr 172.21.15.64:0/6318858 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688effd640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68900717d0 0x7f689015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688f7fe640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6890143c40 0x7f68900c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688e7fc640 1 -- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6890143c40 msgr2=0x7f68900c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688e7fc640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6890143c40 0x7f68900c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688e7fc640 1 -- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68900717d0 msgr2=0x7f689015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688e7fc640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68900717d0 0x7f689015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688e7fc640 1 -- 172.21.15.64:0/6318858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68800094d0 con 0x7f68900ccc10 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688effd640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68900717d0 0x7f689015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688e7fc640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68900ccc10 0x7f68900c6020 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f687c00c3e0 tx=0x7f687c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f68877fe640 1 -- 172.21.15.64:0/6318858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0074e0 con 0x7f68900ccc10 2023-11-24T06:29:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f68877fe640 1 -- 172.21.15.64:0/6318858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c007640 con 0x7f68900ccc10 2023-11-24T06:29:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f68877fe640 1 -- 172.21.15.64:0/6318858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0049e0 con 0x7f68900ccc10 2023-11-24T06:29:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.973+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68900c51f0 con 0x7f68900ccc10 2023-11-24T06:29:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.974+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68900c57b0 con 0x7f68900ccc10 2023-11-24T06:29:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.975+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6890067450 con 0x7f68900ccc10 2023-11-24T06:29:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.976+0000 7f68877fe640 1 -- 172.21.15.64:0/6318858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f687c004b40 con 0x7f68900ccc10 2023-11-24T06:29:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.977+0000 7f68877fe640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f687806ae70 0x7f687806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.977+0000 7f68877fe640 1 -- 172.21.15.64:0/6318858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f687c088fe0 con 0x7f68900ccc10 2023-11-24T06:29:27.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.977+0000 7f688effd640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f687806ae70 0x7f687806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.978+0000 7f688effd640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f687806ae70 0x7f687806d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6880036000 tx=0x7f688003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:27.981+0000 7f68877fe640 1 -- 172.21.15.64:0/6318858 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f687c054080 con 0x7f68900ccc10 2023-11-24T06:29:28.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.178+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68900ceb30 con 0x7f68900ccc10 2023-11-24T06:29:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.180+0000 7f68877fe640 1 -- 172.21.15.64:0/6318858 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f687c0576a0 con 0x7f68900ccc10 2023-11-24T06:29:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:28.185 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:12.271627+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.182+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f687806ae70 msgr2=0x7f687806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.182+0000 7f688ffff640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f687806ae70 0x7f687806d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6880036000 tx=0x7f688003c040 comp rx=0 tx=0).stop 2023-11-24T06:29:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.182+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68900ccc10 msgr2=0x7f68900c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.182+0000 7f688ffff640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68900ccc10 0x7f68900c6020 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f687c00c3e0 tx=0x7f687c00c8a0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 shutdown_connections 2023-11-24T06:29:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f687806ae70 0x7f687806d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6890143c40 0x7f68900c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68900ccc10 0x7f68900c6020 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 --2- 172.21.15.64:0/6318858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68900717d0 0x7f689015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 >> 172.21.15.64:0/6318858 conn(0x7f689006a800 msgr2=0x7f68900cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 shutdown_connections 2023-11-24T06:29:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.183+0000 7f688ffff640 1 -- 172.21.15.64:0/6318858 wait complete. 2023-11-24T06:29:28.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:08.540135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:09.752538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:28.206 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:28.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.434+0000 7f198423d640 1 Processor -- start 2023-11-24T06:29:28.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.435+0000 7f198423d640 1 -- start start 2023-11-24T06:29:28.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.436+0000 7f198423d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f197c13e360 0x7f197c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.436+0000 7f198423d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f197c15f9c0 0x7f197c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.436+0000 7f198423d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c15c5f0 con 0x7f197c13df80 2023-11-24T06:29:28.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.436+0000 7f198423d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c06c470 con 0x7f197c13e360 2023-11-24T06:29:28.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.436+0000 7f198423d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c06c750 con 0x7f197c15f9c0 2023-11-24T06:29:28.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19817d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f197c13e360 0x7f197c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19817d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f197c13e360 0x7f197c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37472/0 (socket says 172.21.15.64:37472) 2023-11-24T06:29:28.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19817d8640 1 -- 172.21.15.64:0/2054899210 learned_addr learned my addr 172.21.15.64:0/2054899210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:28.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19827da640 1 --2- 172.21.15.64:0/2054899210 >> v2:172.21.15.162:3300/0 conn(0x7f197c15f9c0 0x7f197c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19817d8640 1 -- 172.21.15.64:0/2054899210 >> v2:172.21.15.162:3300/0 conn(0x7f197c15f9c0 msgr2=0x7f197c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19817d8640 1 --2- 172.21.15.64:0/2054899210 >> v2:172.21.15.162:3300/0 conn(0x7f197c15f9c0 0x7f197c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19817d8640 1 -- 172.21.15.64:0/2054899210 >> v1:172.21.15.162:6789/0 conn(0x7f197c13df80 legacy=0x7f197c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.437+0000 7f19817d8640 1 -- 172.21.15.64:0/2054899210 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c06ca30 con 0x7f197c13e360 2023-11-24T06:29:28.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.439+0000 7f19817d8640 1 --2- 172.21.15.64:0/2054899210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c13e360 0x7f197c0cf290 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f197c06ded0 tx=0x7f196c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b643ca6162ac88 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.439+0000 7f1980fd7640 1 -- 172.21.15.64:0/2054899210 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c031640 con 0x7f197c13e360 2023-11-24T06:29:28.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.439+0000 7f1980fd7640 1 -- 172.21.15.64:0/2054899210 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c0317a0 con 0x7f197c13e360 2023-11-24T06:29:28.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.439+0000 7f1980fd7640 1 -- 172.21.15.64:0/2054899210 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c031b30 con 0x7f197c13e360 2023-11-24T06:29:28.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 -- 172.21.15.64:0/2054899210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c13e360 msgr2=0x7f197c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 --2- 172.21.15.64:0/2054899210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c13e360 0x7f197c0cf290 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f197c06ded0 tx=0x7f196c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 -- 172.21.15.64:0/2054899210 shutdown_connections 2023-11-24T06:29:28.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 --2- 172.21.15.64:0/2054899210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c13e360 0x7f197c0cf290 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 --2- 172.21.15.64:0/2054899210 >> v2:172.21.15.162:3300/0 conn(0x7f197c15f9c0 0x7f197c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 -- 172.21.15.64:0/2054899210 >> 172.21.15.64:0/2054899210 conn(0x7f197c06a800 msgr2=0x7f197c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 -- 172.21.15.64:0/2054899210 shutdown_connections 2023-11-24T06:29:28.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.440+0000 7f198423d640 1 -- 172.21.15.64:0/2054899210 wait complete. 2023-11-24T06:29:28.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.442+0000 7f198423d640 1 Processor -- start 2023-11-24T06:29:28.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.442+0000 7f198423d640 1 -- start start 2023-11-24T06:29:28.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.443+0000 7f198423d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197c13df80 0x7f197c13d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.443+0000 7f198423d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f197c13e360 0x7f197c15e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.443+0000 7f198423d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c15f9c0 0x7f197c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.443+0000 7f198423d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c14c3a0 con 0x7f197c15f9c0 2023-11-24T06:29:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.443+0000 7f198423d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c13dda0 con 0x7f197c13df80 2023-11-24T06:29:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.443+0000 7f198423d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c14b790 con 0x7f197c13e360 2023-11-24T06:29:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.443+0000 7f19827da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c15f9c0 0x7f197c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c15f9c0 0x7f197c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37474/0 (socket says 172.21.15.64:37474) 2023-11-24T06:29:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 -- 172.21.15.64:0/679226352 learned_addr learned my addr 172.21.15.64:0/679226352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:28.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f1981fd9640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197c13df80 0x7f197c13d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19817d8640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f197c13e360 0x7f197c15e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 -- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f197c13e360 msgr2=0x7f197c15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f197c13e360 0x7f197c15e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 -- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197c13df80 msgr2=0x7f197c13d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197c13df80 0x7f197c13d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 -- 172.21.15.64:0/679226352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0094d0 con 0x7f197c15f9c0 2023-11-24T06:29:28.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.444+0000 7f19827da640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c15f9c0 0x7f197c15a180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1974007e30 tx=0x7f197400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.445+0000 7f19727fc640 1 -- 172.21.15.64:0/679226352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974014620 con 0x7f197c15f9c0 2023-11-24T06:29:28.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.445+0000 7f19727fc640 1 -- 172.21.15.64:0/679226352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197400ee90 con 0x7f197c15f9c0 2023-11-24T06:29:28.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.445+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c14ba70 con 0x7f197c15f9c0 2023-11-24T06:29:28.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.445+0000 7f19727fc640 1 -- 172.21.15.64:0/679226352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740149c0 con 0x7f197c15f9c0 2023-11-24T06:29:28.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.445+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c14c030 con 0x7f197c15f9c0 2023-11-24T06:29:28.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.446+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c14c550 con 0x7f197c15f9c0 2023-11-24T06:29:28.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.447+0000 7f19727fc640 1 -- 172.21.15.64:0/679226352 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1974014b20 con 0x7f197c15f9c0 2023-11-24T06:29:28.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.448+0000 7f19727fc640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f196006ae40 0x7f196006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.448+0000 7f19727fc640 1 -- 172.21.15.64:0/679226352 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f197408a0a0 con 0x7f197c15f9c0 2023-11-24T06:29:28.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.448+0000 7f1981fd9640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f196006ae40 0x7f196006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.449+0000 7f1981fd9640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f196006ae40 0x7f196006d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1978006fd0 tx=0x7f1978008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.453+0000 7f19727fc640 1 -- 172.21.15.64:0/679226352 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f197c14c550 con 0x7f197c15f9c0 2023-11-24T06:29:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.647+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f197c14c550 con 0x7f197c15f9c0 2023-11-24T06:29:28.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.648+0000 7f19727fc640 1 -- 172.21.15.64:0/679226352 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f197c14c550 con 0x7f197c15f9c0 2023-11-24T06:29:28.652 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:29:28.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f196006ae40 msgr2=0x7f196006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f196006ae40 0x7f196006d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1978006fd0 tx=0x7f1978008040 comp rx=0 tx=0).stop 2023-11-24T06:29:28.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c15f9c0 msgr2=0x7f197c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c15f9c0 0x7f197c15a180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1974007e30 tx=0x7f197400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 shutdown_connections 2023-11-24T06:29:28.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f196006ae40 0x7f196006d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f197c15f9c0 0x7f197c15a180 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f197c13e360 0x7f197c15e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 --2- 172.21.15.64:0/679226352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197c13df80 0x7f197c13d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.651+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 >> 172.21.15.64:0/679226352 conn(0x7f197c06a800 msgr2=0x7f197c154460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.652+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 shutdown_connections 2023-11-24T06:29:28.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.652+0000 7f198423d640 1 -- 172.21.15.64:0/679226352 wait complete. 2023-11-24T06:29:28.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:29:28.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.905+0000 7f64bca90640 1 Processor -- start 2023-11-24T06:29:28.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.905+0000 7f64bca90640 1 -- start start 2023-11-24T06:29:28.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.906+0000 7f64bca90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 0x7f64b80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.906+0000 7f64bca90640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b80b6750 con 0x7f64b80ce6f0 2023-11-24T06:29:28.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.906+0000 7f64bca90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b80cf1f0 con 0x7f64b815d590 2023-11-24T06:29:28.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.906+0000 7f64bca90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80cef10 con 0x7f64b80d0280 2023-11-24T06:29:28.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.907+0000 7f64b6d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f64b80ce6f0 0x7f64b80b6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57626/0 (socket says 172.21.15.64:57626) 2023-11-24T06:29:28.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.907+0000 7f64b6d9d640 1 -- 172.21.15.64:0/1108435898 learned_addr learned my addr 172.21.15.64:0/1108435898 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:28.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.907+0000 7f64affff640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 0x7f64b80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.907+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881208337 0 0) 0x7f64b80b6750 con 0x7f64b80ce6f0 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.907+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a4003520 con 0x7f64b80ce6f0 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4097830894 0 0) 0x7f64a4003520 con 0x7f64b80ce6f0 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 msgr2=0x7f64b80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 0x7f64b80ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 >> v1:172.21.15.162:6789/0 conn(0x7f64b815d590 legacy=0x7f64b80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80b6100 con 0x7f64b80ce6f0 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f64b00019f0 con 0x7f64b80ce6f0 2023-11-24T06:29:28.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 >> v1:172.21.15.64:6789/0 conn(0x7f64b80ce6f0 legacy=0x7f64b80b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64a40039a0 0x7f64a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 0x7f64a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a4004d00 0x7f64a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80cef10 con 0x7f64a4004d00 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a400ba30 con 0x7f64a40039a0 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a400bbf0 con 0x7f64b80d0280 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b6d9d640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a4004d00 0x7f64a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64affff640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 0x7f64a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b659c640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64a40039a0 0x7f64a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b6d9d640 1 -- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 msgr2=0x7f64a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b6d9d640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 0x7f64a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b6d9d640 1 -- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64a40039a0 msgr2=0x7f64a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.908+0000 7f64b6d9d640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64a40039a0 0x7f64a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.909+0000 7f64b6d9d640 1 -- 172.21.15.64:0/1108435898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a4003520 con 0x7f64a4004d00 2023-11-24T06:29:28.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.909+0000 7f64affff640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64b80d0280 0x7f64a40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:28.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.909+0000 7f64b659c640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64a40039a0 0x7f64a4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:28.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64b6d9d640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a4004d00 0x7f64a4005180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f64b000a800 tx=0x7f64a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ace3b43482a7a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b00072d0 con 0x7f64a4004d00 2023-11-24T06:29:28.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b002f070 con 0x7f64a4004d00 2023-11-24T06:29:28.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64b5d9b640 1 -- 172.21.15.64:0/1108435898 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b00076b0 con 0x7f64a4004d00 2023-11-24T06:29:28.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64bca90640 1 -- 172.21.15.64:0/1108435898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a4004d00 msgr2=0x7f64a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64bca90640 1 --2- 172.21.15.64:0/1108435898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a4004d00 0x7f64a4005180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f64b000a800 tx=0x7f64a40057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64b6d9d640 1 -- 172.21.15.64:0/1108435898 reap_dead start 2023-11-24T06:29:28.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64bca90640 1 -- 172.21.15.64:0/1108435898 shutdown_connections 2023-11-24T06:29:28.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.910+0000 7f64bca90640 1 -- 172.21.15.64:0/1108435898 >> 172.21.15.64:0/1108435898 conn(0x7f64b809c540 msgr2=0x7f64b80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.911+0000 7f64bca90640 1 -- 172.21.15.64:0/1108435898 shutdown_connections 2023-11-24T06:29:28.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.911+0000 7f64bca90640 1 -- 172.21.15.64:0/1108435898 wait complete. 2023-11-24T06:29:28.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.912+0000 7f64bca90640 1 Processor -- start 2023-11-24T06:29:28.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.913+0000 7f64bca90640 1 -- start start 2023-11-24T06:29:28.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64bca90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64a4004d00 0x7f64b8150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64bca90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b80b3b80 0x7f64b806b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64bca90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b81597b0 0x7f64b806b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64bca90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b36f0 con 0x7f64b80b3b80 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64bca90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b2ae0 con 0x7f64b81597b0 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64bca90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b2dc0 con 0x7f64a4004d00 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b80b3b80 0x7f64b806b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b80b3b80 0x7f64b806b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37492/0 (socket says 172.21.15.64:37492) 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64affff640 1 -- 172.21.15.64:0/800420150 learned_addr learned my addr 172.21.15.64:0/800420150 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64b659c640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64a4004d00 0x7f64b8150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64b6d9d640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b81597b0 0x7f64b806b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.914+0000 7f64affff640 1 -- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64a4004d00 msgr2=0x7f64b8150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64affff640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64a4004d00 0x7f64b8150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64affff640 1 -- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b81597b0 msgr2=0x7f64b806b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64affff640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b81597b0 0x7f64b806b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64affff640 1 -- 172.21.15.64:0/800420150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80b30a0 con 0x7f64b80b3b80 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64b6d9d640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b81597b0 0x7f64b806b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64affff640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b80b3b80 0x7f64b806b010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f64a00108b0 tx=0x7f64a0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64aeffd640 1 -- 172.21.15.64:0/800420150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a000eb60 con 0x7f64b80b3b80 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64aeffd640 1 -- 172.21.15.64:0/800420150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a000ecc0 con 0x7f64b80b3b80 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b80b3380 con 0x7f64b80b3b80 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64aeffd640 1 -- 172.21.15.64:0/800420150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a001c400 con 0x7f64b80b3b80 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.915+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b8159e50 con 0x7f64b80b3b80 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.917+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b80b38a0 con 0x7f64b80b3b80 2023-11-24T06:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.918+0000 7f64aeffd640 1 -- 172.21.15.64:0/800420150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64a001c560 con 0x7f64b80b3b80 2023-11-24T06:29:28.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.918+0000 7f64aeffd640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f649806ae40 0x7f649806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.919+0000 7f64aeffd640 1 -- 172.21.15.64:0/800420150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64a0089bb0 con 0x7f64b80b3b80 2023-11-24T06:29:28.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.919+0000 7f64b659c640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f649806ae40 0x7f649806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.919+0000 7f64b659c640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f649806ae40 0x7f649806d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f64b80b0130 tx=0x7f64a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:28.923+0000 7f64aeffd640 1 -- 172.21.15.64:0/800420150 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64b80b38a0 con 0x7f64b80b3b80 2023-11-24T06:29:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.118+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64b8066d60 con 0x7f64b80b3b80 2023-11-24T06:29:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.119+0000 7f64aeffd640 1 -- 172.21.15.64:0/800420150 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64a0054da0 con 0x7f64b80b3b80 2023-11-24T06:29:29.124 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:29:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f649806ae40 msgr2=0x7f649806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f649806ae40 0x7f649806d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f64b80b0130 tx=0x7f64a8009040 comp rx=0 tx=0).stop 2023-11-24T06:29:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b80b3b80 msgr2=0x7f64b806b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b80b3b80 0x7f64b806b010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f64a00108b0 tx=0x7f64a0010d70 comp rx=0 tx=0).stop 2023-11-24T06:29:29.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 shutdown_connections 2023-11-24T06:29:29.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f649806ae40 0x7f649806d300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b81597b0 0x7f64b806b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b80b3b80 0x7f64b806b010 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 --2- 172.21.15.64:0/800420150 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64a4004d00 0x7f64b8150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.122+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 >> 172.21.15.64:0/800420150 conn(0x7f64b809c540 msgr2=0x7f64b815c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.123+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 shutdown_connections 2023-11-24T06:29:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.123+0000 7f64bca90640 1 -- 172.21.15.64:0/800420150 wait complete. 2023-11-24T06:29:29.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:29:29.137 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:29:29.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:29:29.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.401+0000 7fadb9691640 1 Processor -- start 2023-11-24T06:29:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.402+0000 7fadb9691640 1 -- start start 2023-11-24T06:29:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.403+0000 7fadb9691640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.403+0000 7fadb9691640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fadb40cf9a0 0x7fadb40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.403+0000 7fadb9691640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fadb40cddf0 0x7fadb40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.403+0000 7fadb9691640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb40d0320 con 0x7fadb40cddf0 2023-11-24T06:29:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.403+0000 7fadb9691640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb40b2730 con 0x7fadb40cf9a0 2023-11-24T06:29:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.403+0000 7fadb9691640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40d1ec0 con 0x7fadb415d4b0 2023-11-24T06:29:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fadb40cddf0 0x7fadb40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fadb40cddf0 0x7fadb40ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37504/0 (socket says 172.21.15.64:37504) 2023-11-24T06:29:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 -- 172.21.15.64:0/2753080488 learned_addr learned my addr 172.21.15.64:0/2753080488 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb37fe640 1 --2- 172.21.15.64:0/2753080488 >> v2:172.21.15.162:3300/0 conn(0x7fadb40cf9a0 0x7fadb40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb3fff640 1 --2- 172.21.15.64:0/2753080488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 -- 172.21.15.64:0/2753080488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 msgr2=0x7fadb40d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 --2- 172.21.15.64:0/2753080488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 -- 172.21.15.64:0/2753080488 >> v2:172.21.15.162:3300/0 conn(0x7fadb40cf9a0 msgr2=0x7fadb40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 --2- 172.21.15.64:0/2753080488 >> v2:172.21.15.162:3300/0 conn(0x7fadb40cf9a0 0x7fadb40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb8e90640 1 -- 172.21.15.64:0/2753080488 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb40b29d0 con 0x7fadb40cddf0 2023-11-24T06:29:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.404+0000 7fadb37fe640 1 --2- 172.21.15.64:0/2753080488 >> v2:172.21.15.162:3300/0 conn(0x7fadb40cf9a0 0x7fadb40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.406+0000 7fadb8e90640 1 --2- 172.21.15.64:0/2753080488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40ce280 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fadac009e50 tx=0x7fadac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc99859f33c45f51 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.406+0000 7fadb2ffd640 1 -- 172.21.15.64:0/2753080488 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac02ca70 con 0x7fadb40cddf0 2023-11-24T06:29:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.406+0000 7fadb2ffd640 1 -- 172.21.15.64:0/2753080488 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac02cbd0 con 0x7fadb40cddf0 2023-11-24T06:29:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.406+0000 7fadb2ffd640 1 -- 172.21.15.64:0/2753080488 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0343f0 con 0x7fadb40cddf0 2023-11-24T06:29:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.406+0000 7fadb9691640 1 -- 172.21.15.64:0/2753080488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 msgr2=0x7fadb40ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.406+0000 7fadb9691640 1 --2- 172.21.15.64:0/2753080488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40ce280 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fadac009e50 tx=0x7fadac02bed0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.407+0000 7fadb9691640 1 -- 172.21.15.64:0/2753080488 shutdown_connections 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.407+0000 7fadb9691640 1 --2- 172.21.15.64:0/2753080488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40ce280 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.407+0000 7fadb9691640 1 --2- 172.21.15.64:0/2753080488 >> v2:172.21.15.162:3300/0 conn(0x7fadb40cf9a0 0x7fadb40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.407+0000 7fadb9691640 1 --2- 172.21.15.64:0/2753080488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.407+0000 7fadb9691640 1 -- 172.21.15.64:0/2753080488 >> 172.21.15.64:0/2753080488 conn(0x7fadb40d3000 msgr2=0x7fadb40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.407+0000 7fadb9691640 1 -- 172.21.15.64:0/2753080488 shutdown_connections 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.407+0000 7fadb9691640 1 -- 172.21.15.64:0/2753080488 wait complete. 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.409+0000 7fadb9691640 1 Processor -- start 2023-11-24T06:29:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.409+0000 7fadb9691640 1 -- start start 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb9691640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40b8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb9691640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadb40cf9a0 0x7fadb4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb9691640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb9691640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40b7970 con 0x7fadb40cddf0 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb9691640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40b6b20 con 0x7fadb40cf9a0 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb9691640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40b6df0 con 0x7fadb415d4b0 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40b8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37518/0 (socket says 172.21.15.64:37518) 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.410+0000 7fadb3fff640 1 -- 172.21.15.64:0/506239286 learned_addr learned my addr 172.21.15.64:0/506239286 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb37fe640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadb40cf9a0 0x7fadb4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb3fff640 1 -- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 msgr2=0x7fadb40b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb8e90640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb3fff640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb3fff640 1 -- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadb40cf9a0 msgr2=0x7fadb4159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb3fff640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadb40cf9a0 0x7fadb4159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb3fff640 1 -- 172.21.15.64:0/506239286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac009b30 con 0x7fadb40cddf0 2023-11-24T06:29:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb37fe640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadb40cf9a0 0x7fadb4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb3fff640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40b8290 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fada400d800 tx=0x7fada400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb0ff9640 1 -- 172.21.15.64:0/506239286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada400b6c0 con 0x7fadb40cddf0 2023-11-24T06:29:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb0ff9640 1 -- 172.21.15.64:0/506239286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada400b820 con 0x7fadb40cddf0 2023-11-24T06:29:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb415da10 con 0x7fadb40cddf0 2023-11-24T06:29:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb0ff9640 1 -- 172.21.15.64:0/506239286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada400bbb0 con 0x7fadb40cddf0 2023-11-24T06:29:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.411+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb40b73b0 con 0x7fadb40cddf0 2023-11-24T06:29:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.413+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb4067450 con 0x7fadb40cddf0 2023-11-24T06:29:29.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.413+0000 7fadb0ff9640 1 -- 172.21.15.64:0/506239286 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fada400bd10 con 0x7fadb40cddf0 2023-11-24T06:29:29.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.414+0000 7fadb0ff9640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fada006ae40 0x7fada006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.414+0000 7fadb0ff9640 1 -- 172.21.15.64:0/506239286 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fada4089c00 con 0x7fadb40cddf0 2023-11-24T06:29:29.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.415+0000 7fadb37fe640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fada006ae40 0x7fada006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.415+0000 7fadb37fe640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fada006ae40 0x7fada006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fada8006fd0 tx=0x7fada8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.419+0000 7fadb0ff9640 1 -- 172.21.15.64:0/506239286 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fada4090080 con 0x7fadb40cddf0 2023-11-24T06:29:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.614+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadb4159720 con 0x7fadb40cddf0 2023-11-24T06:29:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.615+0000 7fadb0ff9640 1 -- 172.21.15.64:0/506239286 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fada4054cb0 con 0x7fadb40cddf0 2023-11-24T06:29:29.618 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:29:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fada006ae40 msgr2=0x7fada006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fada006ae40 0x7fada006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fada8006fd0 tx=0x7fada8008040 comp rx=0 tx=0).stop 2023-11-24T06:29:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 msgr2=0x7fadb40b8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40b8290 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fada400d800 tx=0x7fada400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 shutdown_connections 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fada006ae40 0x7fada006d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadb415d4b0 0x7fadb40b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadb40cf9a0 0x7fadb4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 --2- 172.21.15.64:0/506239286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb40cddf0 0x7fadb40b8290 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.617+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 >> 172.21.15.64:0/506239286 conn(0x7fadb40d3000 msgr2=0x7fadb415c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.618+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 shutdown_connections 2023-11-24T06:29:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.618+0000 7fadb9691640 1 -- 172.21.15.64:0/506239286 wait complete. 2023-11-24T06:29:29.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:29:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.860+0000 7fc94efa9640 1 Processor -- start 2023-11-24T06:29:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.860+0000 7fc94efa9640 1 -- start start 2023-11-24T06:29:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.862+0000 7fc94efa9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc948150760 0x7fc948072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.862+0000 7fc94efa9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc94815d300 0x7fc94815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.862+0000 7fc94efa9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94815dc80 con 0x7fc9480b2bd0 2023-11-24T06:29:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.862+0000 7fc94efa9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9480b3770 con 0x7fc948150760 2023-11-24T06:29:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.862+0000 7fc94efa9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc948071190 con 0x7fc94815d300 2023-11-24T06:29:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc94cd45640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc948150760 0x7fc948072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc94cd45640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc948150760 0x7fc948072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37534/0 (socket says 172.21.15.64:37534) 2023-11-24T06:29:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc94cd45640 1 -- 172.21.15.64:0/3731229123 learned_addr learned my addr 172.21.15.64:0/3731229123 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc93ffff640 1 --2- 172.21.15.64:0/3731229123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc94815d300 0x7fc94815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc94cd45640 1 -- 172.21.15.64:0/3731229123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc94815d300 msgr2=0x7fc94815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc94cd45640 1 --2- 172.21.15.64:0/3731229123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc94815d300 0x7fc94815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc94cd45640 1 -- 172.21.15.64:0/3731229123 >> v1:172.21.15.162:6789/0 conn(0x7fc9480b2bd0 legacy=0x7fc9480b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.863+0000 7fc94cd45640 1 -- 172.21.15.64:0/3731229123 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948071470 con 0x7fc948150760 2023-11-24T06:29:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.865+0000 7fc94cd45640 1 --2- 172.21.15.64:0/3731229123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc948150760 0x7fc948072230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc944009e50 tx=0x7fc94402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd88c29d055243 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.865+0000 7fc93f7fe640 1 -- 172.21.15.64:0/3731229123 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94402cb00 con 0x7fc948150760 2023-11-24T06:29:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.865+0000 7fc93f7fe640 1 -- 172.21.15.64:0/3731229123 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94402cc60 con 0x7fc948150760 2023-11-24T06:29:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.865+0000 7fc93f7fe640 1 -- 172.21.15.64:0/3731229123 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9440343f0 con 0x7fc948150760 2023-11-24T06:29:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 -- 172.21.15.64:0/3731229123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc948150760 msgr2=0x7fc948072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 --2- 172.21.15.64:0/3731229123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc948150760 0x7fc948072230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc944009e50 tx=0x7fc94402bf60 comp rx=0 tx=0).stop 2023-11-24T06:29:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 -- 172.21.15.64:0/3731229123 shutdown_connections 2023-11-24T06:29:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 --2- 172.21.15.64:0/3731229123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc94815d300 0x7fc94815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 --2- 172.21.15.64:0/3731229123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc948150760 0x7fc948072230 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 -- 172.21.15.64:0/3731229123 >> 172.21.15.64:0/3731229123 conn(0x7fc94806a800 msgr2=0x7fc94806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 -- 172.21.15.64:0/3731229123 shutdown_connections 2023-11-24T06:29:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.866+0000 7fc94efa9640 1 -- 172.21.15.64:0/3731229123 wait complete. 2023-11-24T06:29:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.868+0000 7fc94efa9640 1 Processor -- start 2023-11-24T06:29:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.868+0000 7fc94efa9640 1 -- start start 2023-11-24T06:29:29.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.869+0000 7fc94efa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9480b2bd0 0x7fc9481453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.869+0000 7fc94efa9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc948150760 0x7fc948141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.869+0000 7fc94efa9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc94815d300 0x7fc948141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94efa9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9481578a0 con 0x7fc9480b2bd0 2023-11-24T06:29:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94efa9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815a880 con 0x7fc94815d300 2023-11-24T06:29:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94efa9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc948141fb0 con 0x7fc948150760 2023-11-24T06:29:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9480b2bd0 0x7fc9481453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9480b2bd0 0x7fc9481453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37536/0 (socket says 172.21.15.64:37536) 2023-11-24T06:29:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 -- 172.21.15.64:0/765036153 learned_addr learned my addr 172.21.15.64:0/765036153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:29.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 -- 172.21.15.64:0/765036153 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc948150760 msgr2=0x7fc948141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc948150760 0x7fc948141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 -- 172.21.15.64:0/765036153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc94815d300 msgr2=0x7fc948141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc94815d300 0x7fc948141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.870+0000 7fc94cd45640 1 -- 172.21.15.64:0/765036153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944009b30 con 0x7fc9480b2bd0 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.871+0000 7fc94cd45640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9480b2bd0 0x7fc9481453c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc944009e50 tx=0x7fc944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.871+0000 7fc93d7fa640 1 -- 172.21.15.64:0/765036153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944009510 con 0x7fc9480b2bd0 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.871+0000 7fc93d7fa640 1 -- 172.21.15.64:0/765036153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94403e450 con 0x7fc9480b2bd0 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.871+0000 7fc93d7fa640 1 -- 172.21.15.64:0/765036153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94403e800 con 0x7fc9480b2bd0 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.871+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc948159cb0 con 0x7fc9480b2bd0 2023-11-24T06:29:29.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.871+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94815a230 con 0x7fc9480b2bd0 2023-11-24T06:29:29.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.873+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9480675d0 con 0x7fc9480b2bd0 2023-11-24T06:29:29.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.873+0000 7fc93d7fa640 1 -- 172.21.15.64:0/765036153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc94403e960 con 0x7fc9480b2bd0 2023-11-24T06:29:29.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.874+0000 7fc93d7fa640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc93406ae70 0x7fc93406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.874+0000 7fc93d7fa640 1 -- 172.21.15.64:0/765036153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9440ac1a0 con 0x7fc9480b2bd0 2023-11-24T06:29:29.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.875+0000 7fc93ffff640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc93406ae70 0x7fc93406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.876+0000 7fc93ffff640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc93406ae70 0x7fc93406d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc938006fd0 tx=0x7fc938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:29.879+0000 7fc93d7fa640 1 -- 172.21.15.64:0/765036153 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc944077300 con 0x7fc9480b2bd0 2023-11-24T06:29:30.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.092+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc948070780 con 0x7fc9480b2bd0 2023-11-24T06:29:30.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.093+0000 7fc93d7fa640 1 -- 172.21.15.64:0/765036153 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc94407a920 con 0x7fc9480b2bd0 2023-11-24T06:29:30.097 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T06:29:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.095+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc93406ae70 msgr2=0x7fc93406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.095+0000 7fc94efa9640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc93406ae70 0x7fc93406d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc938006fd0 tx=0x7fc938008040 comp rx=0 tx=0).stop 2023-11-24T06:29:30.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.095+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9480b2bd0 msgr2=0x7fc9481453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.095+0000 7fc94efa9640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9480b2bd0 0x7fc9481453c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc944009e50 tx=0x7fc944004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:30.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 shutdown_connections 2023-11-24T06:29:30.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc93406ae70 0x7fc93406d330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc94815d300 0x7fc948141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc948150760 0x7fc948141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 --2- 172.21.15.64:0/765036153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9480b2bd0 0x7fc9481453c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 >> 172.21.15.64:0/765036153 conn(0x7fc94806a800 msgr2=0x7fc9480afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 shutdown_connections 2023-11-24T06:29:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.096+0000 7fc94efa9640 1 -- 172.21.15.64:0/765036153 wait complete. 2023-11-24T06:29:30.119 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2023-11-24T06:29:30.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:30.119 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:30.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[95590]: fuse finished with error 0 and tester_r 0 2023-11-24T06:29:30.222 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:29:30.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:30.223 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:29:30.379 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:29:30.379 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:30.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:29:30.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:29:30.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:30.444 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:29:30.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:29:30.507 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:29:30.508 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:29:30.508 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:30.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:29:30.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:29:30.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:30.569 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:29:30.623 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:29:30.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:29:30.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:30.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.854+0000 7f085acd1640 1 Processor -- start 2023-11-24T06:29:30.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.855+0000 7f085acd1640 1 -- start start 2023-11-24T06:29:30.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.856+0000 7f085acd1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08540d0100 0x7f08540b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.856+0000 7f085acd1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08540d0f40 0x7f08540d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.856+0000 7f085acd1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08540d0a10 con 0x7f0854160920 2023-11-24T06:29:30.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.856+0000 7f085acd1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08540cf080 con 0x7f08540d0100 2023-11-24T06:29:30.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.856+0000 7f085acd1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08540c6be0 con 0x7f08540d0f40 2023-11-24T06:29:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.857+0000 7f085a4d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08540d0f40 0x7f08540d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.857+0000 7f08594ce640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08540d0100 0x7f08540b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.857+0000 7f085a4d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08540d0f40 0x7f08540d13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:33792/0 (socket says 172.21.15.64:33792) 2023-11-24T06:29:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.857+0000 7f085a4d0640 1 -- 172.21.15.64:0/4178784037 learned_addr learned my addr 172.21.15.64:0/4178784037 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.858+0000 7f085a4d0640 1 -- 172.21.15.64:0/4178784037 >> v2:172.21.15.162:3300/0 conn(0x7f08540d0100 msgr2=0x7f08540b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.858+0000 7f085a4d0640 1 --2- 172.21.15.64:0/4178784037 >> v2:172.21.15.162:3300/0 conn(0x7f08540d0100 0x7f08540b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.858+0000 7f085a4d0640 1 -- 172.21.15.64:0/4178784037 >> v1:172.21.15.162:6789/0 conn(0x7f0854160920 legacy=0x7f08540cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.858+0000 7f085a4d0640 1 -- 172.21.15.64:0/4178784037 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08540ce7e0 con 0x7f08540d0f40 2023-11-24T06:29:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.859+0000 7f085a4d0640 1 --2- 172.21.15.64:0/4178784037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08540d0f40 0x7f08540d13d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f084c009e50 tx=0x7f084c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c7ec55cfba4f8f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f0858ccd640 1 -- 172.21.15.64:0/4178784037 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c02cb00 con 0x7f08540d0f40 2023-11-24T06:29:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f0858ccd640 1 -- 172.21.15.64:0/4178784037 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c02cc60 con 0x7f08540d0f40 2023-11-24T06:29:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f0858ccd640 1 -- 172.21.15.64:0/4178784037 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c0343f0 con 0x7f08540d0f40 2023-11-24T06:29:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f085acd1640 1 -- 172.21.15.64:0/4178784037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08540d0f40 msgr2=0x7f08540d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f085acd1640 1 --2- 172.21.15.64:0/4178784037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08540d0f40 0x7f08540d13d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f084c009e50 tx=0x7f084c02bf60 comp rx=0 tx=0).stop 2023-11-24T06:29:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f085acd1640 1 -- 172.21.15.64:0/4178784037 shutdown_connections 2023-11-24T06:29:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f085acd1640 1 --2- 172.21.15.64:0/4178784037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08540d0f40 0x7f08540d13d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f085acd1640 1 --2- 172.21.15.64:0/4178784037 >> v2:172.21.15.162:3300/0 conn(0x7f08540d0100 0x7f08540b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.860+0000 7f085acd1640 1 -- 172.21.15.64:0/4178784037 >> 172.21.15.64:0/4178784037 conn(0x7f08540d3090 msgr2=0x7f08540d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:30.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.861+0000 7f085acd1640 1 -- 172.21.15.64:0/4178784037 shutdown_connections 2023-11-24T06:29:30.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.861+0000 7f085acd1640 1 -- 172.21.15.64:0/4178784037 wait complete. 2023-11-24T06:29:30.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.863+0000 7f085acd1640 1 Processor -- start 2023-11-24T06:29:30.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.863+0000 7f085acd1640 1 -- start start 2023-11-24T06:29:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f085acd1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08540d0100 0x7f08541f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f085acd1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08540d0f40 0x7f08540c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f085acd1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0854160920 0x7f08540c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f085acd1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085415a360 con 0x7f08540d0f40 2023-11-24T06:29:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f085acd1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854159400 con 0x7f08540d0100 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f085acd1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08541596e0 con 0x7f0854160920 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f08594ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08540d0f40 0x7f08540c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f08594ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08540d0f40 0x7f08540c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37552/0 (socket says 172.21.15.64:37552) 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f08594ce640 1 -- 172.21.15.64:0/1592047925 learned_addr learned my addr 172.21.15.64:0/1592047925 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f085a4d0640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0854160920 0x7f08540c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.865+0000 7f0859ccf640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08540d0100 0x7f08541f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f08594ce640 1 -- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0854160920 msgr2=0x7f08540c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f08594ce640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0854160920 0x7f08540c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f08594ce640 1 -- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08540d0100 msgr2=0x7f08541f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f08594ce640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08540d0100 0x7f08541f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f08594ce640 1 -- 172.21.15.64:0/1592047925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c009b30 con 0x7f08540d0f40 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f085a4d0640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0854160920 0x7f08540c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f08594ce640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08540d0f40 0x7f08540c7e60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f084400e870 tx=0x7f084400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f084a7fc640 1 -- 172.21.15.64:0/1592047925 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08440094f0 con 0x7f08540d0f40 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f084a7fc640 1 -- 172.21.15.64:0/1592047925 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084400c940 con 0x7f08540d0f40 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08541599c0 con 0x7f08540d0f40 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f084a7fc640 1 -- 172.21.15.64:0/1592047925 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084400ccf0 con 0x7f08540d0f40 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.866+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0854159f80 con 0x7f08540d0f40 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.868+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08540d3b60 con 0x7f08540d0f40 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.869+0000 7f084a7fc640 1 -- 172.21.15.64:0/1592047925 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0844013470 con 0x7f08540d0f40 2023-11-24T06:29:30.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.870+0000 7f084a7fc640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f083806ae40 0x7f083806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.870+0000 7f084a7fc640 1 -- 172.21.15.64:0/1592047925 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08440896a0 con 0x7f08540d0f40 2023-11-24T06:29:30.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.870+0000 7f0859ccf640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f083806ae40 0x7f083806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.871+0000 7f0859ccf640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f083806ae40 0x7f083806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f08500015a0 tx=0x7f0850009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:30.874+0000 7f084a7fc640 1 -- 172.21.15.64:0/1592047925 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08440547e0 con 0x7f08540d0f40 2023-11-24T06:29:31.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.108+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0854067450 con 0x7f08540d0f40 2023-11-24T06:29:31.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.109+0000 7f084a7fc640 1 -- 172.21.15.64:0/1592047925 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f0844057e00 con 0x7f08540d0f40 2023-11-24T06:29:31.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:31.113 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3831306721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3831306721},{"type":"v1","addr":"172.21.15.162:6834","nonce":3831306721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1401622125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1401622125},{"type":"v1","addr":"172.21.15.162:6835","nonce":1401622125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2378368513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6835","nonce":2378368513},{"type":"v1","addr":"172.21.15.64:6837","nonce":2378368513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:10.724367+0000","modified":"2023-11-24T06:29:13.855414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6836/2541249514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2541249514},{"type":"v1","addr":"172.21.15.64:6836","nonce":2541249514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:29:31.114 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T06:29:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f083806ae40 msgr2=0x7f083806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f083806ae40 0x7f083806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f08500015a0 tx=0x7f0850009040 comp rx=0 tx=0).stop 2023-11-24T06:29:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08540d0f40 msgr2=0x7f08540c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08540d0f40 0x7f08540c7e60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f084400e870 tx=0x7f084400ed30 comp rx=0 tx=0).stop 2023-11-24T06:29:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 shutdown_connections 2023-11-24T06:29:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f083806ae40 0x7f083806d300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0854160920 0x7f08540c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08540d0f40 0x7f08540c7e60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 --2- 172.21.15.64:0/1592047925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08540d0100 0x7f08541f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 >> 172.21.15.64:0/1592047925 conn(0x7f08540d3090 msgr2=0x7f08540d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 shutdown_connections 2023-11-24T06:29:31.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.112+0000 7f085acd1640 1 -- 172.21.15.64:0/1592047925 wait complete. 2023-11-24T06:29:31.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:31.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.356+0000 7f66d1813640 1 Processor -- start 2023-11-24T06:29:31.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.356+0000 7f66d1813640 1 -- start start 2023-11-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.358+0000 7f66d1813640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 0x7f66cc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.358+0000 7f66d1813640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66cc071f00 0x7f66cc166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.358+0000 7f66d1813640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc0b2e10 con 0x7f66cc06c7c0 2023-11-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.358+0000 7f66d1813640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66cc06cc20 con 0x7f66cc071f00 2023-11-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.358+0000 7f66d1813640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc06cec0 con 0x7f66cc071410 2023-11-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.358+0000 7f66cbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f66cc06c7c0 0x7f66cc0717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57640/0 (socket says 172.21.15.64:57640) 2023-11-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.358+0000 7f66cbfff640 1 -- 172.21.15.64:0/1651611469 learned_addr learned my addr 172.21.15.64:0/1651611469 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66d0811640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 0x7f66cc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66d1012640 1 --2- 172.21.15.64:0/1651611469 >> v2:172.21.15.162:3300/0 conn(0x7f66cc071f00 0x7f66cc166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316078965 0 0) 0x7f66cc0b2e10 con 0x7f66cc06c7c0 2023-11-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66bc003520 con 0x7f66cc06c7c0 2023-11-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3986277090 0 0) 0x7f66bc003520 con 0x7f66cc06c7c0 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 msgr2=0x7f66cc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 0x7f66cc146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 >> v2:172.21.15.162:3300/0 conn(0x7f66cc071f00 msgr2=0x7f66cc166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 --2- 172.21.15.64:0/1651611469 >> v2:172.21.15.162:3300/0 conn(0x7f66cc071f00 0x7f66cc166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc06d1a0 con 0x7f66cc06c7c0 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f66b8001670 con 0x7f66cc06c7c0 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 >> v1:172.21.15.64:6789/0 conn(0x7f66cc06c7c0 legacy=0x7f66cc0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.359+0000 7f66cb7fe640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc003970 0x7f66bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cb7fe640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc0047c0 0x7f66bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cb7fe640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 0x7f66bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc06cc20 con 0x7f66bc0047c0 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc06cec0 con 0x7f66bc003970 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc00ba30 con 0x7f66cc071410 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cbfff640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc0047c0 0x7f66bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66d0811640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc003970 0x7f66bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66d1012640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 0x7f66bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cbfff640 1 -- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 msgr2=0x7f66bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cbfff640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66cc071410 0x7f66bc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cbfff640 1 -- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc003970 msgr2=0x7f66bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cbfff640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc003970 0x7f66bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66cbfff640 1 -- 172.21.15.64:0/1651611469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc003520 con 0x7f66bc0047c0 2023-11-24T06:29:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.360+0000 7f66d0811640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc003970 0x7f66bc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:31.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.362+0000 7f66cbfff640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc0047c0 0x7f66bc004c40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c210 tx=0x7f66bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643c2f55459fdca1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.362+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8005ad0 con 0x7f66bc0047c0 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.362+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8005c30 con 0x7f66bc0047c0 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.362+0000 7f66cb7fe640 1 -- 172.21.15.64:0/1651611469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b802c8a0 con 0x7f66bc0047c0 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.363+0000 7f66d1813640 1 -- 172.21.15.64:0/1651611469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc0047c0 msgr2=0x7f66bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.363+0000 7f66d1813640 1 --2- 172.21.15.64:0/1651611469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc0047c0 0x7f66bc004c40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c210 tx=0x7f66bc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.363+0000 7f66d1012640 1 -- 172.21.15.64:0/1651611469 reap_dead start 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.363+0000 7f66d1813640 1 -- 172.21.15.64:0/1651611469 shutdown_connections 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.363+0000 7f66d1813640 1 -- 172.21.15.64:0/1651611469 >> 172.21.15.64:0/1651611469 conn(0x7f66cc06a800 msgr2=0x7f66cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.363+0000 7f66d1813640 1 -- 172.21.15.64:0/1651611469 shutdown_connections 2023-11-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.363+0000 7f66d1813640 1 -- 172.21.15.64:0/1651611469 wait complete. 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.365+0000 7f66d1813640 1 Processor -- start 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.365+0000 7f66d1813640 1 -- start start 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1813640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc0047c0 0x7f66cc1e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1813640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1e8830 0x7f66cc12e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1813640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66cc12ea90 0x7f66cc12eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1813640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc1e8c90 con 0x7f66cc12ea90 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1813640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc12c560 con 0x7f66cc1e8830 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1813640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc12c7d0 con 0x7f66bc0047c0 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1012640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66cc12ea90 0x7f66cc12eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1012640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66cc12ea90 0x7f66cc12eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37574/0 (socket says 172.21.15.64:37574) 2023-11-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.366+0000 7f66d1012640 1 -- 172.21.15.64:0/1328656545 learned_addr learned my addr 172.21.15.64:0/1328656545 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66cbfff640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1e8830 0x7f66cc12e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d0811640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc0047c0 0x7f66cc1e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1012640 1 -- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc0047c0 msgr2=0x7f66cc1e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1012640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc0047c0 0x7f66cc1e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1012640 1 -- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1e8830 msgr2=0x7f66cc12e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1012640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1e8830 0x7f66cc12e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1012640 1 -- 172.21.15.64:0/1328656545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc12cab0 con 0x7f66cc12ea90 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1012640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66cc12ea90 0x7f66cc12eef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0adf40 tx=0x7f66c4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66c97fa640 1 -- 172.21.15.64:0/1328656545 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4003ea0 con 0x7f66cc12ea90 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66c97fa640 1 -- 172.21.15.64:0/1328656545 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c400f050 con 0x7f66cc12ea90 2023-11-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc12cd90 con 0x7f66cc12ea90 2023-11-24T06:29:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66c97fa640 1 -- 172.21.15.64:0/1328656545 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4014650 con 0x7f66cc12ea90 2023-11-24T06:29:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.367+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc1e4f40 con 0x7f66cc12ea90 2023-11-24T06:29:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.369+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66cc067450 con 0x7f66cc12ea90 2023-11-24T06:29:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.370+0000 7f66c97fa640 1 -- 172.21.15.64:0/1328656545 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66c40147b0 con 0x7f66cc12ea90 2023-11-24T06:29:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.371+0000 7f66c97fa640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f66b006ae40 0x7f66b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.371+0000 7f66c97fa640 1 -- 172.21.15.64:0/1328656545 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66c4089aa0 con 0x7f66cc12ea90 2023-11-24T06:29:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.371+0000 7f66d0811640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f66b006ae40 0x7f66b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.372+0000 7f66d0811640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f66b006ae40 0x7f66b006d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66c0006fd0 tx=0x7f66c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.375+0000 7f66c97fa640 1 -- 172.21.15.64:0/1328656545 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66c4054be0 con 0x7f66cc12ea90 2023-11-24T06:29:31.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.570+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66cc0b41e0 con 0x7f66cc12ea90 2023-11-24T06:29:31.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.573+0000 7f66c97fa640 1 -- 172.21.15.64:0/1328656545 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f66c4058200 con 0x7f66cc12ea90 2023-11-24T06:29:31.577 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:31.577 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:12.271627+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:31.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.575+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f66b006ae40 msgr2=0x7f66b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.575+0000 7f66d1813640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f66b006ae40 0x7f66b006d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66c0006fd0 tx=0x7f66c0008040 comp rx=0 tx=0).stop 2023-11-24T06:29:31.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.575+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66cc12ea90 msgr2=0x7f66cc12eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.575+0000 7f66d1813640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66cc12ea90 0x7f66cc12eef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0adf40 tx=0x7f66c4007700 comp rx=0 tx=0).stop 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 shutdown_connections 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f66b006ae40 0x7f66b006d300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66cc12ea90 0x7f66cc12eef0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1e8830 0x7f66cc12e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 --2- 172.21.15.64:0/1328656545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc0047c0 0x7f66cc1e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 >> 172.21.15.64:0/1328656545 conn(0x7f66cc06a800 msgr2=0x7f66cc0af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 shutdown_connections 2023-11-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.576+0000 7f66d1813640 1 -- 172.21.15.64:0/1328656545 wait complete. 2023-11-24T06:29:31.599 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:08.540135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:09.752538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:31.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.813+0000 7fdd1b5dc640 1 Processor -- start 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.813+0000 7fdd1b5dc640 1 -- start start 2023-11-24T06:29:31.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.815+0000 7fdd1b5dc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdd140ae5a0 0x7fdd1415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.815+0000 7fdd1b5dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd140b3ae0 0x7fdd1406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.815+0000 7fdd1b5dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd1406d120 0x7fdd140cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.815+0000 7fdd1b5dc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd14071450 con 0x7fdd1406d120 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.815+0000 7fdd1b5dc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd14070c40 con 0x7fdd140ae5a0 2023-11-24T06:29:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.815+0000 7fdd1b5dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1406c470 con 0x7fdd140b3ae0 2023-11-24T06:29:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd1406d120 0x7fdd140cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd1406d120 0x7fdd140cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37578/0 (socket says 172.21.15.64:37578) 2023-11-24T06:29:31.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 -- 172.21.15.64:0/2751782493 learned_addr learned my addr 172.21.15.64:0/2751782493 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:31.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd18b77640 1 --2- 172.21.15.64:0/2751782493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd140b3ae0 0x7fdd1406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19378640 1 --2- 172.21.15.64:0/2751782493 >> v2:172.21.15.162:3300/0 conn(0x7fdd140ae5a0 0x7fdd1415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 -- 172.21.15.64:0/2751782493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd140b3ae0 msgr2=0x7fdd1406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 --2- 172.21.15.64:0/2751782493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd140b3ae0 0x7fdd1406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 -- 172.21.15.64:0/2751782493 >> v2:172.21.15.162:3300/0 conn(0x7fdd140ae5a0 msgr2=0x7fdd1415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 --2- 172.21.15.64:0/2751782493 >> v2:172.21.15.162:3300/0 conn(0x7fdd140ae5a0 0x7fdd1415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.816+0000 7fdd19b79640 1 -- 172.21.15.64:0/2751782493 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1406c750 con 0x7fdd1406d120 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.817+0000 7fdd19b79640 1 --2- 172.21.15.64:0/2751782493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1406d120 0x7fdd140cf290 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdd140cfe20 tx=0x7fdd0c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad238b7f3cc4301 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd0bfff640 1 -- 172.21.15.64:0/2751782493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c02ca70 con 0x7fdd1406d120 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd0bfff640 1 -- 172.21.15.64:0/2751782493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c02cbd0 con 0x7fdd1406d120 2023-11-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd0bfff640 1 -- 172.21.15.64:0/2751782493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0343f0 con 0x7fdd1406d120 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/2751782493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1406d120 msgr2=0x7fdd140cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/2751782493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1406d120 0x7fdd140cf290 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdd140cfe20 tx=0x7fdd0c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/2751782493 shutdown_connections 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/2751782493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1406d120 0x7fdd140cf290 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/2751782493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd140b3ae0 0x7fdd1406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/2751782493 >> v2:172.21.15.162:3300/0 conn(0x7fdd140ae5a0 0x7fdd1415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/2751782493 >> 172.21.15.64:0/2751782493 conn(0x7fdd1406a800 msgr2=0x7fdd1406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/2751782493 shutdown_connections 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.818+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/2751782493 wait complete. 2023-11-24T06:29:31.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.820+0000 7fdd1b5dc640 1 Processor -- start 2023-11-24T06:29:31.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.820+0000 7fdd1b5dc640 1 -- start start 2023-11-24T06:29:31.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.821+0000 7fdd1b5dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd1406d120 0x7fdd141f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.821+0000 7fdd1b5dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd140ae5a0 0x7fdd140ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.821+0000 7fdd1b5dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd140b3ae0 0x7fdd140c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.821+0000 7fdd1b5dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cd100 con 0x7fdd140b3ae0 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.821+0000 7fdd1b5dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cd3e0 con 0x7fdd140ae5a0 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.821+0000 7fdd1b5dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cd6c0 con 0x7fdd1406d120 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd140b3ae0 0x7fdd140c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd140b3ae0 0x7fdd140c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37580/0 (socket says 172.21.15.64:37580) 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 -- 172.21.15.64:0/336038624 learned_addr learned my addr 172.21.15.64:0/336038624 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19378640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd1406d120 0x7fdd141f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd18b77640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd140ae5a0 0x7fdd140ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 -- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd1406d120 msgr2=0x7fdd141f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd1406d120 0x7fdd141f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 -- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd140ae5a0 msgr2=0x7fdd140ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd140ae5a0 0x7fdd140ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 -- 172.21.15.64:0/336038624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c009b30 con 0x7fdd140b3ae0 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19378640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd1406d120 0x7fdd141f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.822+0000 7fdd19b79640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd140b3ae0 0x7fdd140c4d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009f40 tx=0x7fdd0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.823+0000 7fdd09ffb640 1 -- 172.21.15.64:0/336038624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c009510 con 0x7fdd140b3ae0 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.823+0000 7fdd09ffb640 1 -- 172.21.15.64:0/336038624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c03e450 con 0x7fdd140b3ae0 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.823+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd140cd9a0 con 0x7fdd140b3ae0 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.823+0000 7fdd09ffb640 1 -- 172.21.15.64:0/336038624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c03e7e0 con 0x7fdd140b3ae0 2023-11-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.823+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd140cdf60 con 0x7fdd140b3ae0 2023-11-24T06:29:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.824+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd140ce150 con 0x7fdd140b3ae0 2023-11-24T06:29:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.825+0000 7fdd09ffb640 1 -- 172.21.15.64:0/336038624 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd0c03e940 con 0x7fdd140b3ae0 2023-11-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.826+0000 7fdd09ffb640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcf806ae90 0x7fdcf806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.826+0000 7fdd09ffb640 1 -- 172.21.15.64:0/336038624 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd0c0ac380 con 0x7fdd140b3ae0 2023-11-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.826+0000 7fdd19378640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcf806ae90 0x7fdcf806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.827+0000 7fdd19378640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcf806ae90 0x7fdcf806d350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006fd0 tx=0x7fdd10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:31.831+0000 7fdd09ffb640 1 -- 172.21.15.64:0/336038624 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd0c0774a0 con 0x7fdd140b3ae0 2023-11-24T06:29:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.028+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd140675d0 con 0x7fdd140b3ae0 2023-11-24T06:29:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.029+0000 7fdd09ffb640 1 -- 172.21.15.64:0/336038624 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd0c07aac0 con 0x7fdd140b3ae0 2023-11-24T06:29:32.033 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:29:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcf806ae90 msgr2=0x7fdcf806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcf806ae90 0x7fdcf806d350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006fd0 tx=0x7fdd10008040 comp rx=0 tx=0).stop 2023-11-24T06:29:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd140b3ae0 msgr2=0x7fdd140c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd140b3ae0 0x7fdd140c4d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009f40 tx=0x7fdd0c004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 shutdown_connections 2023-11-24T06:29:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fdcf806ae90 0x7fdcf806d350 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd140b3ae0 0x7fdd140c4d50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd140ae5a0 0x7fdd140ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 --2- 172.21.15.64:0/336038624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd1406d120 0x7fdd141f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.031+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 >> 172.21.15.64:0/336038624 conn(0x7fdd1406a800 msgr2=0x7fdd140b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.032+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 shutdown_connections 2023-11-24T06:29:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.032+0000 7fdd1b5dc640 1 -- 172.21.15.64:0/336038624 wait complete. 2023-11-24T06:29:32.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.280+0000 7fce9d3a7640 1 Processor -- start 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.281+0000 7fce9d3a7640 1 -- start start 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.282+0000 7fce9d3a7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce980b3b20 0x7fce9815a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.282+0000 7fce9d3a7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce980b3fb0 con 0x7fce981505b0 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.282+0000 7fce9d3a7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce980b1e30 con 0x7fce9806d6d0 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.282+0000 7fce9d3a7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9815abd0 con 0x7fce980b3b20 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.283+0000 7fce97fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fce981505b0 0x7fce9806cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57646/0 (socket says 172.21.15.64:57646) 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.283+0000 7fce97fff640 1 -- 172.21.15.64:0/1140569964 learned_addr learned my addr 172.21.15.64:0/1140569964 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.283+0000 7fce9cba6640 1 --2- 172.21.15.64:0/1140569964 >> v2:172.21.15.64:3300/0 conn(0x7fce980b3b20 0x7fce9815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.283+0000 7fce977fe640 1 -- 172.21.15.64:0/1140569964 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275095464 0 0) 0x7fce980b3fb0 con 0x7fce981505b0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.283+0000 7fce9cba6640 1 -- 172.21.15.64:0/1140569964 >> v1:172.21.15.162:6789/0 conn(0x7fce9806d6d0 legacy=0x7fce980ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.283+0000 7fce9cba6640 1 -- 172.21.15.64:0/1140569964 >> v1:172.21.15.64:6789/0 conn(0x7fce981505b0 legacy=0x7fce9806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.283+0000 7fce9cba6640 1 -- 172.21.15.64:0/1140569964 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980b20a0 con 0x7fce980b3b20 2023-11-24T06:29:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.284+0000 7fce9cba6640 1 --2- 172.21.15.64:0/1140569964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce9815a690 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fce9000aea0 tx=0x7fce9002cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5967a885e390fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce977fe640 1 -- 172.21.15.64:0/1140569964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9002db60 con 0x7fce980b3b20 2023-11-24T06:29:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce977fe640 1 -- 172.21.15.64:0/1140569964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9002dcc0 con 0x7fce980b3b20 2023-11-24T06:29:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce977fe640 1 -- 172.21.15.64:0/1140569964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90008660 con 0x7fce980b3b20 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce9d3a7640 1 -- 172.21.15.64:0/1140569964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 msgr2=0x7fce9815a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/1140569964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce9815a690 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fce9000aea0 tx=0x7fce9002cfc0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce9d3a7640 1 -- 172.21.15.64:0/1140569964 shutdown_connections 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/1140569964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce9815a690 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.285+0000 7fce9d3a7640 1 -- 172.21.15.64:0/1140569964 >> 172.21.15.64:0/1140569964 conn(0x7fce9806a7d0 msgr2=0x7fce9806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.286+0000 7fce9d3a7640 1 -- 172.21.15.64:0/1140569964 shutdown_connections 2023-11-24T06:29:32.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.286+0000 7fce9d3a7640 1 -- 172.21.15.64:0/1140569964 wait complete. 2023-11-24T06:29:32.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.287+0000 7fce9d3a7640 1 Processor -- start 2023-11-24T06:29:32.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.288+0000 7fce9d3a7640 1 -- start start 2023-11-24T06:29:32.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.288+0000 7fce9d3a7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce9806d6d0 0x7fce9813e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce9d3a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce98150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce9d3a7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce981505b0 0x7fce9813ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce9d3a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce981e6540 con 0x7fce980b3b20 2023-11-24T06:29:32.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce9d3a7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce981e70c0 con 0x7fce9806d6d0 2023-11-24T06:29:32.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce9d3a7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9813eb70 con 0x7fce981505b0 2023-11-24T06:29:32.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce98150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce98150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37596/0 (socket says 172.21.15.64:37596) 2023-11-24T06:29:32.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 -- 172.21.15.64:0/4165866089 learned_addr learned my addr 172.21.15.64:0/4165866089 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:32.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce9cba6640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce981505b0 0x7fce9813ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 -- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce981505b0 msgr2=0x7fce9813ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce981505b0 0x7fce9813ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce97fff640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce9806d6d0 0x7fce9813e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 -- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce9806d6d0 msgr2=0x7fce9813e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce9806d6d0 0x7fce9813e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce8ffff640 1 -- 172.21.15.64:0/4165866089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9000ab80 con 0x7fce980b3b20 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.289+0000 7fce97fff640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce9806d6d0 0x7fce9813e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.290+0000 7fce8ffff640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce98150c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fce8000b3f0 tx=0x7fce8000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.290+0000 7fce957fa640 1 -- 172.21.15.64:0/4165866089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80011830 con 0x7fce980b3b20 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.290+0000 7fce957fa640 1 -- 172.21.15.64:0/4165866089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce80011990 con 0x7fce980b3b20 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.290+0000 7fce957fa640 1 -- 172.21.15.64:0/4165866089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80011d40 con 0x7fce980b3b20 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.290+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce981e6720 con 0x7fce980b3b20 2023-11-24T06:29:32.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.290+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce981e6ce0 con 0x7fce980b3b20 2023-11-24T06:29:32.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.291+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce981e7200 con 0x7fce980b3b20 2023-11-24T06:29:32.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.292+0000 7fce957fa640 1 -- 172.21.15.64:0/4165866089 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce8001a400 con 0x7fce980b3b20 2023-11-24T06:29:32.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.293+0000 7fce957fa640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7806ae70 0x7fce7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.293+0000 7fce957fa640 1 -- 172.21.15.64:0/4165866089 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce80089420 con 0x7fce980b3b20 2023-11-24T06:29:32.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.294+0000 7fce97fff640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7806ae70 0x7fce7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.294+0000 7fce97fff640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7806ae70 0x7fce7806d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce88000f60 tx=0x7fce88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.298+0000 7fce957fa640 1 -- 172.21.15.64:0/4165866089 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce800544d0 con 0x7fce980b3b20 2023-11-24T06:29:32.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.491+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce98067450 con 0x7fce980b3b20 2023-11-24T06:29:32.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.492+0000 7fce957fa640 1 -- 172.21.15.64:0/4165866089 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce80057af0 con 0x7fce980b3b20 2023-11-24T06:29:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T06:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.495+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7806ae70 msgr2=0x7fce7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.495+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7806ae70 0x7fce7806d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce88000f60 tx=0x7fce88009040 comp rx=0 tx=0).stop 2023-11-24T06:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.495+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 msgr2=0x7fce98150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.495+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce98150c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fce8000b3f0 tx=0x7fce8000b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 shutdown_connections 2023-11-24T06:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce7806ae70 0x7fce7806d330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce981505b0 0x7fce9813ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce980b3b20 0x7fce98150c50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 --2- 172.21.15.64:0/4165866089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce9806d6d0 0x7fce9813e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 >> 172.21.15.64:0/4165866089 conn(0x7fce9806a7d0 msgr2=0x7fce9814f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 shutdown_connections 2023-11-24T06:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.496+0000 7fce9d3a7640 1 -- 172.21.15.64:0/4165866089 wait complete. 2023-11-24T06:29:32.519 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.741+0000 7fc5249ac640 1 Processor -- start 2023-11-24T06:29:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.742+0000 7fc5249ac640 1 -- start start 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.743+0000 7fc5249ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc520143810 0x7fc520071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.743+0000 7fc5249ac640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc52015d770 0x7fc52015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.743+0000 7fc5249ac640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52015e140 con 0x7fc5200ae170 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.743+0000 7fc5249ac640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc520143bf0 con 0x7fc52015d770 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc5249ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52006c730 con 0x7fc520143810 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc52015d770 0x7fc52015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc52015d770 0x7fc52015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37608/0 (socket says 172.21.15.64:37608) 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51ffff640 1 -- 172.21.15.64:0/3449401876 learned_addr learned my addr 172.21.15.64:0/3449401876 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51f7fe640 1 --2- 172.21.15.64:0/3449401876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc520143810 0x7fc520071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51ffff640 1 -- 172.21.15.64:0/3449401876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc520143810 msgr2=0x7fc520071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51ffff640 1 --2- 172.21.15.64:0/3449401876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc520143810 0x7fc520071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51ffff640 1 -- 172.21.15.64:0/3449401876 >> v1:172.21.15.64:6789/0 conn(0x7fc5200ae170 legacy=0x7fc52015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51ffff640 1 -- 172.21.15.64:0/3449401876 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5200ae5f0 con 0x7fc52015d770 2023-11-24T06:29:32.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51e7fc640 1 -- 172.21.15.64:0/3449401876 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661129515 0 0) 0x7fc52015e140 con 0x7fc5200ae170 2023-11-24T06:29:32.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.744+0000 7fc51f7fe640 1 --2- 172.21.15.64:0/3449401876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc520143810 0x7fc520071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:32.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.746+0000 7fc51ffff640 1 --2- 172.21.15.64:0/3449401876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc52015d770 0x7fc52015dc00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc518009870 tx=0x7fc51802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e207394d08708a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.746+0000 7fc51e7fc640 1 -- 172.21.15.64:0/3449401876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180317c0 con 0x7fc52015d770 2023-11-24T06:29:32.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.746+0000 7fc51e7fc640 1 -- 172.21.15.64:0/3449401876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518031920 con 0x7fc52015d770 2023-11-24T06:29:32.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.746+0000 7fc51e7fc640 1 -- 172.21.15.64:0/3449401876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518031cb0 con 0x7fc52015d770 2023-11-24T06:29:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 -- 172.21.15.64:0/3449401876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc52015d770 msgr2=0x7fc52015dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 --2- 172.21.15.64:0/3449401876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc52015d770 0x7fc52015dc00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc518009870 tx=0x7fc51802b860 comp rx=0 tx=0).stop 2023-11-24T06:29:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 -- 172.21.15.64:0/3449401876 shutdown_connections 2023-11-24T06:29:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 --2- 172.21.15.64:0/3449401876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc52015d770 0x7fc52015dc00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 --2- 172.21.15.64:0/3449401876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc520143810 0x7fc520071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 -- 172.21.15.64:0/3449401876 >> 172.21.15.64:0/3449401876 conn(0x7fc52006a7d0 msgr2=0x7fc52006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 -- 172.21.15.64:0/3449401876 shutdown_connections 2023-11-24T06:29:32.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.747+0000 7fc5249ac640 1 -- 172.21.15.64:0/3449401876 wait complete. 2023-11-24T06:29:32.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.749+0000 7fc5249ac640 1 Processor -- start 2023-11-24T06:29:32.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.749+0000 7fc5249ac640 1 -- start start 2023-11-24T06:29:32.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc5249ac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200ae170 0x7fc52015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc5249ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc520143810 0x7fc5200ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc5249ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc52015d770 0x7fc52013ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc5249ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52015c3f0 con 0x7fc520143810 2023-11-24T06:29:32.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc5249ac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5200ce750 con 0x7fc5200ae170 2023-11-24T06:29:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc5249ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5200cc690 con 0x7fc52015d770 2023-11-24T06:29:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc520143810 0x7fc5200ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc520143810 0x7fc5200ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37612/0 (socket says 172.21.15.64:37612) 2023-11-24T06:29:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 -- 172.21.15.64:0/680016901 learned_addr learned my addr 172.21.15.64:0/680016901 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:32.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51ffff640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc52015d770 0x7fc52013ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51f7fe640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200ae170 0x7fc52015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 -- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc52015d770 msgr2=0x7fc52013ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc52015d770 0x7fc52013ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 -- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200ae170 msgr2=0x7fc52015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200ae170 0x7fc52015beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.750+0000 7fc51effd640 1 -- 172.21.15.64:0/680016901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180094d0 con 0x7fc520143810 2023-11-24T06:29:32.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.751+0000 7fc51effd640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc520143810 0x7fc5200ce210 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c4e0 tx=0x7fc50800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.751+0000 7fc517fff640 1 -- 172.21.15.64:0/680016901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508005480 con 0x7fc520143810 2023-11-24T06:29:32.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.751+0000 7fc517fff640 1 -- 172.21.15.64:0/680016901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5080035f0 con 0x7fc520143810 2023-11-24T06:29:32.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.751+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5200cc970 con 0x7fc520143810 2023-11-24T06:29:32.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.751+0000 7fc517fff640 1 -- 172.21.15.64:0/680016901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5080039c0 con 0x7fc520143810 2023-11-24T06:29:32.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.751+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5201f2a80 con 0x7fc520143810 2023-11-24T06:29:32.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.753+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5201f2c70 con 0x7fc520143810 2023-11-24T06:29:32.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.753+0000 7fc517fff640 1 -- 172.21.15.64:0/680016901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc508003b20 con 0x7fc520143810 2023-11-24T06:29:32.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.755+0000 7fc517fff640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc50006ae40 0x7fc50006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.755+0000 7fc517fff640 1 -- 172.21.15.64:0/680016901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc508088b10 con 0x7fc520143810 2023-11-24T06:29:32.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.755+0000 7fc51f7fe640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc50006ae40 0x7fc50006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.756+0000 7fc51f7fe640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc50006ae40 0x7fc50006d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5200cd3f0 tx=0x7fc510003a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.759+0000 7fc517fff640 1 -- 172.21.15.64:0/680016901 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc508053c60 con 0x7fc520143810 2023-11-24T06:29:32.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.954+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc520067450 con 0x7fc520143810 2023-11-24T06:29:32.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.955+0000 7fc517fff640 1 -- 172.21.15.64:0/680016901 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc508017070 con 0x7fc520143810 2023-11-24T06:29:32.959 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T06:29:32.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc50006ae40 msgr2=0x7fc50006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc50006ae40 0x7fc50006d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5200cd3f0 tx=0x7fc510003a10 comp rx=0 tx=0).stop 2023-11-24T06:29:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc520143810 msgr2=0x7fc5200ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc520143810 0x7fc5200ce210 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c4e0 tx=0x7fc50800c9a0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 shutdown_connections 2023-11-24T06:29:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc50006ae40 0x7fc50006d300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc52015d770 0x7fc52013ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc520143810 0x7fc5200ce210 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 --2- 172.21.15.64:0/680016901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200ae170 0x7fc52015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.957+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 >> 172.21.15.64:0/680016901 conn(0x7fc52006a7d0 msgr2=0x7fc5200b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.958+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 shutdown_connections 2023-11-24T06:29:32.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:32.958+0000 7fc5249ac640 1 -- 172.21.15.64:0/680016901 wait complete. 2023-11-24T06:29:32.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:33.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.202+0000 7fb38210c640 1 Processor -- start 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.202+0000 7fb38210c640 1 -- start start 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.203+0000 7fb38210c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb37c13df80 0x7fb37c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.203+0000 7fb38210c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb37c0cf9a0 0x7fb37c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.203+0000 7fb38210c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37c06c470 con 0x7fb37c13e360 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.203+0000 7fb38210c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37c06c750 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.203+0000 7fb38210c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37c06ca30 con 0x7fb37c13df80 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb37c0cf9a0 0x7fb37c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb37c0cf9a0 0x7fb37c0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49618/0 (socket says 172.21.15.64:49618) 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37bfff640 1 -- 172.21.15.64:0/2520794156 learned_addr learned my addr 172.21.15.64:0/2520794156 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37b7fe640 1 --2- 172.21.15.64:0/2520794156 >> v2:172.21.15.162:3300/0 conn(0x7fb37c13df80 0x7fb37c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37bfff640 1 -- 172.21.15.64:0/2520794156 >> v2:172.21.15.162:3300/0 conn(0x7fb37c13df80 msgr2=0x7fb37c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37bfff640 1 --2- 172.21.15.64:0/2520794156 >> v2:172.21.15.162:3300/0 conn(0x7fb37c13df80 0x7fb37c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37bfff640 1 -- 172.21.15.64:0/2520794156 >> v1:172.21.15.162:6789/0 conn(0x7fb37c13e360 legacy=0x7fb37c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.204+0000 7fb37bfff640 1 -- 172.21.15.64:0/2520794156 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c06cd10 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.205+0000 7fb37bfff640 1 --2- 172.21.15.64:0/2520794156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c0cd840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb37c072770 tx=0x7fb37402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9687fad718d6b9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.206+0000 7fb37a7fc640 1 -- 172.21.15.64:0/2520794156 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37402ca80 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.206+0000 7fb37a7fc640 1 -- 172.21.15.64:0/2520794156 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37402cbe0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.206+0000 7fb37a7fc640 1 -- 172.21.15.64:0/2520794156 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3740343f0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.206+0000 7fb38210c640 1 -- 172.21.15.64:0/2520794156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 msgr2=0x7fb37c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.206+0000 7fb38210c640 1 --2- 172.21.15.64:0/2520794156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c0cd840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb37c072770 tx=0x7fb37402bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.207+0000 7fb38210c640 1 -- 172.21.15.64:0/2520794156 shutdown_connections 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.207+0000 7fb38210c640 1 --2- 172.21.15.64:0/2520794156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c0cd840 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.207+0000 7fb38210c640 1 --2- 172.21.15.64:0/2520794156 >> v2:172.21.15.162:3300/0 conn(0x7fb37c13df80 0x7fb37c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.207+0000 7fb38210c640 1 -- 172.21.15.64:0/2520794156 >> 172.21.15.64:0/2520794156 conn(0x7fb37c06a800 msgr2=0x7fb37c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.207+0000 7fb38210c640 1 -- 172.21.15.64:0/2520794156 shutdown_connections 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.207+0000 7fb38210c640 1 -- 172.21.15.64:0/2520794156 wait complete. 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.208+0000 7fb38210c640 1 Processor -- start 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.209+0000 7fb38210c640 1 -- start start 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.209+0000 7fb38210c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c12f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.209+0000 7fb38210c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb37c13df80 0x7fb37c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb38210c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb37c13e360 0x7fb37c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb38210c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c130a70 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb38210c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c0ae2f0 con 0x7fb37c13e360 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb38210c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c12fe60 con 0x7fb37c13df80 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c12f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c12f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49626/0 (socket says 172.21.15.64:49626) 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 -- 172.21.15.64:0/257866202 learned_addr learned my addr 172.21.15.64:0/257866202 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37affd640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb37c13df80 0x7fb37c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 -- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb37c13df80 msgr2=0x7fb37c0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb37c13df80 0x7fb37c0addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37bfff640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb37c13e360 0x7fb37c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 -- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb37c13e360 msgr2=0x7fb37c15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb37c13e360 0x7fb37c15df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.210+0000 7fb37b7fe640 1 -- 172.21.15.64:0/257866202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb374009b30 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.211+0000 7fb37b7fe640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c12f830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00d800 tx=0x7fb36c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.211+0000 7fb363fff640 1 -- 172.21.15.64:0/257866202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00b6c0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.211+0000 7fb363fff640 1 -- 172.21.15.64:0/257866202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36c00b820 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.211+0000 7fb363fff640 1 -- 172.21.15.64:0/257866202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00bbb0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.211+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37c130140 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.211+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37c130700 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.212+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37c1308c0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.213+0000 7fb363fff640 1 -- 172.21.15.64:0/257866202 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb36c00bd10 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.214+0000 7fb363fff640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb36806ae70 0x7fb36806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.214+0000 7fb363fff640 1 -- 172.21.15.64:0/257866202 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb36c089bc0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.214+0000 7fb37affd640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb36806ae70 0x7fb36806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.215+0000 7fb37affd640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb36806ae70 0x7fb36806d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb37c13df50 tx=0x7fb370009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.219+0000 7fb363fff640 1 -- 172.21.15.64:0/257866202 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb36c054bd0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.456+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb37c0675d0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.457+0000 7fb363fff640 1 -- 172.21.15.64:0/257866202 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fb36c0581f0 con 0x7fb37c0cf9a0 2023-11-24T06:29:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:33.461 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3831306721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3831306721},{"type":"v1","addr":"172.21.15.162:6834","nonce":3831306721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1401622125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1401622125},{"type":"v1","addr":"172.21.15.162:6835","nonce":1401622125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2378368513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6835","nonce":2378368513},{"type":"v1","addr":"172.21.15.64:6837","nonce":2378368513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:10.724367+0000","modified":"2023-11-24T06:29:13.855414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6836/2541249514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2541249514},{"type":"v1","addr":"172.21.15.64:6836","nonce":2541249514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:29:33.462 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb36806ae70 msgr2=0x7fb36806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb36806ae70 0x7fb36806d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb37c13df50 tx=0x7fb370009040 comp rx=0 tx=0).stop 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 msgr2=0x7fb37c12f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c12f830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00d800 tx=0x7fb36c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 shutdown_connections 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb36806ae70 0x7fb36806d330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb37c13e360 0x7fb37c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb37c13df80 0x7fb37c0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 --2- 172.21.15.64:0/257866202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37c0cf9a0 0x7fb37c12f830 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 >> 172.21.15.64:0/257866202 conn(0x7fb37c06a800 msgr2=0x7fb37c0ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:33.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.460+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 shutdown_connections 2023-11-24T06:29:33.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.461+0000 7fb38210c640 1 -- 172.21.15.64:0/257866202 wait complete. 2023-11-24T06:29:33.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:33.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.708+0000 7f47915db640 1 Processor -- start 2023-11-24T06:29:33.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.709+0000 7f47915db640 1 -- start start 2023-11-24T06:29:33.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.710+0000 7f47915db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c160920 0x7f478c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.710+0000 7f47915db640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f478c0cfe70 0x7f478c161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.710+0000 7f47915db640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c0d0a10 con 0x7f478c0cf9f0 2023-11-24T06:29:33.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.710+0000 7f47915db640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478c0d1f50 con 0x7f478c0cfe70 2023-11-24T06:29:33.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.710+0000 7f47915db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0cf080 con 0x7f478c160920 2023-11-24T06:29:33.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.711+0000 7f4783fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f478c0cf9f0 0x7f478c161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43706/0 (socket says 172.21.15.64:43706) 2023-11-24T06:29:33.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.711+0000 7f4783fff640 1 -- 172.21.15.64:0/2535763116 learned_addr learned my addr 172.21.15.64:0/2535763116 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:33.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.711+0000 7f478bfff640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c160920 0x7f478c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.711+0000 7f4790dda640 1 --2- 172.21.15.64:0/2535763116 >> v2:172.21.15.162:3300/0 conn(0x7f478c0cfe70 0x7f478c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.711+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914546666 0 0) 0x7f478c0d0a10 con 0x7f478c0cf9f0 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.711+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4778003520 con 0x7f478c0cf9f0 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1526818159 0 0) 0x7f4778003520 con 0x7f478c0cf9f0 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c160920 msgr2=0x7f478c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c160920 0x7f478c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 >> v2:172.21.15.162:3300/0 conn(0x7f478c0cfe70 msgr2=0x7f478c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 --2- 172.21.15.64:0/2535763116 >> v2:172.21.15.162:3300/0 conn(0x7f478c0cfe70 0x7f478c161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0c6be0 con 0x7f478c0cf9f0 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f4774001670 con 0x7f478c0cf9f0 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 >> v1:172.21.15.64:6789/0 conn(0x7f478c0cf9f0 legacy=0x7f478c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4778003970 0x7f4778003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f4778004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c160920 0x7f4778005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0d1f50 con 0x7f47780047c0 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0cf080 con 0x7f4778003970 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477800b9d0 con 0x7f478c160920 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f4790dda640 1 --2- 172.21.15.64:0/2535763116 >> v2:172.21.15.162:3300/0 conn(0x7f478c0cfe70 0x7f478c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.712+0000 7f4783fff640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f4778004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.713+0000 7f4783fff640 1 -- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c160920 msgr2=0x7f4778005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.713+0000 7f4783fff640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c160920 0x7f4778005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.713+0000 7f4783fff640 1 -- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4778003970 msgr2=0x7f4778003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.713+0000 7f4783fff640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4778003970 0x7f4778003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.713+0000 7f4783fff640 1 -- 172.21.15.64:0/2535763116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778003520 con 0x7f47780047c0 2023-11-24T06:29:33.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.714+0000 7f4783fff640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f4778004c40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f477400c110 tx=0x7f47780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e623c75fd5613e4e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.714+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774005ad0 con 0x7f47780047c0 2023-11-24T06:29:33.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.714+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4774005c50 con 0x7f47780047c0 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.714+0000 7f478b7fe640 1 -- 172.21.15.64:0/2535763116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477402c830 con 0x7f47780047c0 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.715+0000 7f47915db640 1 -- 172.21.15.64:0/2535763116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 msgr2=0x7f4778004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.715+0000 7f47915db640 1 --2- 172.21.15.64:0/2535763116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f4778004c40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f477400c110 tx=0x7f47780057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.715+0000 7f4790dda640 1 -- 172.21.15.64:0/2535763116 reap_dead start 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.715+0000 7f47915db640 1 -- 172.21.15.64:0/2535763116 shutdown_connections 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.715+0000 7f47915db640 1 -- 172.21.15.64:0/2535763116 >> 172.21.15.64:0/2535763116 conn(0x7f478c0d3090 msgr2=0x7f478c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.715+0000 7f47915db640 1 -- 172.21.15.64:0/2535763116 shutdown_connections 2023-11-24T06:29:33.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.715+0000 7f47915db640 1 -- 172.21.15.64:0/2535763116 wait complete. 2023-11-24T06:29:33.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.717+0000 7f47915db640 1 Processor -- start 2023-11-24T06:29:33.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.717+0000 7f47915db640 1 -- start start 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f47915db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f478c1f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f47915db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c0b6d90 0x7f478c0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f47915db640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478c1f3fd0 0x7f478c1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f47915db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0b6970 con 0x7f47780047c0 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f47915db640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0b5cf0 con 0x7f478c1f3fd0 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f47915db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0b5fd0 con 0x7f478c0b6d90 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f478c1f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f478c1f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49646/0 (socket says 172.21.15.64:49646) 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 -- 172.21.15.64:0/3740550334 learned_addr learned my addr 172.21.15.64:0/3740550334 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 -- 172.21.15.64:0/3740550334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c0b6d90 msgr2=0x7f478c0b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c0b6d90 0x7f478c0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 -- 172.21.15.64:0/3740550334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478c1f3fd0 msgr2=0x7f478c1f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478c1f3fd0 0x7f478c1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.718+0000 7f478bfff640 1 -- 172.21.15.64:0/3740550334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0b62b0 con 0x7f47780047c0 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.719+0000 7f478bfff640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f478c1f6c70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f477c00e870 tx=0x7f477c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.719+0000 7f47897fa640 1 -- 172.21.15.64:0/3740550334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00c940 con 0x7f47780047c0 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.719+0000 7f47897fa640 1 -- 172.21.15.64:0/3740550334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c00cac0 con 0x7f47780047c0 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.719+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c0b6590 con 0x7f47780047c0 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.719+0000 7f47897fa640 1 -- 172.21.15.64:0/3740550334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00ce70 con 0x7f47780047c0 2023-11-24T06:29:33.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.719+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c1f7390 con 0x7f47780047c0 2023-11-24T06:29:33.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.720+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c067450 con 0x7f47780047c0 2023-11-24T06:29:33.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.721+0000 7f47897fa640 1 -- 172.21.15.64:0/3740550334 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f477c013470 con 0x7f47780047c0 2023-11-24T06:29:33.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.722+0000 7f47897fa640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f476c06ae70 0x7f476c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.722+0000 7f47897fa640 1 -- 172.21.15.64:0/3740550334 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f477c089590 con 0x7f47780047c0 2023-11-24T06:29:33.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.722+0000 7f4783fff640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f476c06ae70 0x7f476c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.723+0000 7f4783fff640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f476c06ae70 0x7f476c06d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4774034000 tx=0x7f477402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.727+0000 7f47897fa640 1 -- 172.21.15.64:0/3740550334 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f477c054630 con 0x7f47780047c0 2023-11-24T06:29:33.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.919+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f478c08c7e0 con 0x7f47780047c0 2023-11-24T06:29:33.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.924+0000 7f47897fa640 1 -- 172.21.15.64:0/3740550334 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f477c057c50 con 0x7f47780047c0 2023-11-24T06:29:33.928 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:33.928 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:12.271627+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:33.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.926+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f476c06ae70 msgr2=0x7f476c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.926+0000 7f47915db640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f476c06ae70 0x7f476c06d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4774034000 tx=0x7f477402e000 comp rx=0 tx=0).stop 2023-11-24T06:29:33.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.926+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 msgr2=0x7f478c1f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.926+0000 7f47915db640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f478c1f6c70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f477c00e870 tx=0x7f477c00ed30 comp rx=0 tx=0).stop 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 shutdown_connections 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f476c06ae70 0x7f476c06d330 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478c1f3fd0 0x7f478c1f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f478c0b6d90 0x7f478c0b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 --2- 172.21.15.64:0/3740550334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47780047c0 0x7f478c1f6c70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 >> 172.21.15.64:0/3740550334 conn(0x7f478c0d3090 msgr2=0x7f478c0b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 shutdown_connections 2023-11-24T06:29:33.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:33.927+0000 7f47915db640 1 -- 172.21.15.64:0/3740550334 wait complete. 2023-11-24T06:29:33.950 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:29:33.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:34.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.168+0000 7f5163fff640 1 Processor -- start 2023-11-24T06:29:34.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.168+0000 7f5163fff640 1 -- start start 2023-11-24T06:29:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.169+0000 7f5163fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f516415d440 0x7f51640d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.169+0000 7f5163fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f516406c220 0x7f51640cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.169+0000 7f5163fff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51640cfd00 con 0x7f516406b700 2023-11-24T06:29:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.169+0000 7f5163fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51640cfaf0 con 0x7f516406c220 2023-11-24T06:29:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.169+0000 7f5163fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51640cffe0 con 0x7f516415d440 2023-11-24T06:29:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.169+0000 7f51637fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f516406c220 0x7f51640cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.170+0000 7f51637fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f516406c220 0x7f51640cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49662/0 (socket says 172.21.15.64:49662) 2023-11-24T06:29:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.170+0000 7f51637fe640 1 -- 172.21.15.64:0/575175146 learned_addr learned my addr 172.21.15.64:0/575175146 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.170+0000 7f51637fe640 1 -- 172.21.15.64:0/575175146 >> v2:172.21.15.162:3300/0 conn(0x7f516415d440 msgr2=0x7f51640d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.170+0000 7f51637fe640 1 --2- 172.21.15.64:0/575175146 >> v2:172.21.15.162:3300/0 conn(0x7f516415d440 0x7f51640d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.170+0000 7f51637fe640 1 -- 172.21.15.64:0/575175146 >> v1:172.21.15.162:6789/0 conn(0x7f516406b700 legacy=0x7f516406bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.170+0000 7f51637fe640 1 -- 172.21.15.64:0/575175146 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51640d02c0 con 0x7f516406c220 2023-11-24T06:29:34.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.171+0000 7f51637fe640 1 --2- 172.21.15.64:0/575175146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516406c220 0x7f51640cb950 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f51640ce8e0 tx=0x7f515c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc2176da21a7c23 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.171+0000 7f51627fc640 1 -- 172.21.15.64:0/575175146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c031870 con 0x7f516406c220 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.171+0000 7f51627fc640 1 -- 172.21.15.64:0/575175146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c0319d0 con 0x7f516406c220 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.171+0000 7f51627fc640 1 -- 172.21.15.64:0/575175146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c031d60 con 0x7f516406c220 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 -- 172.21.15.64:0/575175146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516406c220 msgr2=0x7f51640cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 --2- 172.21.15.64:0/575175146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516406c220 0x7f51640cb950 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f51640ce8e0 tx=0x7f515c02b910 comp rx=0 tx=0).stop 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 -- 172.21.15.64:0/575175146 shutdown_connections 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 --2- 172.21.15.64:0/575175146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516406c220 0x7f51640cb950 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 --2- 172.21.15.64:0/575175146 >> v2:172.21.15.162:3300/0 conn(0x7f516415d440 0x7f51640d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 -- 172.21.15.64:0/575175146 >> 172.21.15.64:0/575175146 conn(0x7f51640d0de0 msgr2=0x7f51640d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 -- 172.21.15.64:0/575175146 shutdown_connections 2023-11-24T06:29:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.172+0000 7f5163fff640 1 -- 172.21.15.64:0/575175146 wait complete. 2023-11-24T06:29:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.174+0000 7f5163fff640 1 Processor -- start 2023-11-24T06:29:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.174+0000 7f5163fff640 1 -- start start 2023-11-24T06:29:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f5163fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f516406b700 0x7f5164152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f5163fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f516406c220 0x7f516409c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f5163fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516415d440 0x7f516409cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f5163fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51640c5170 con 0x7f516415d440 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f5163fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516409bef0 con 0x7f516406b700 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f5163fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5164152940 con 0x7f516406c220 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f51637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516415d440 0x7f516409cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f51637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516415d440 0x7f516409cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49678/0 (socket says 172.21.15.64:49678) 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.175+0000 7f51637fe640 1 -- 172.21.15.64:0/1491718 learned_addr learned my addr 172.21.15.64:0/1491718 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f5162ffd640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f516406b700 0x7f5164152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f515a7fc640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f516406c220 0x7f516409c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f51637fe640 1 -- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f516406c220 msgr2=0x7f516409c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f51637fe640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f516406c220 0x7f516409c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f51637fe640 1 -- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f516406b700 msgr2=0x7f5164152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f51637fe640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f516406b700 0x7f5164152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f51637fe640 1 -- 172.21.15.64:0/1491718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0094d0 con 0x7f516415d440 2023-11-24T06:29:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f51637fe640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516415d440 0x7f516409cb90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f515c02b950 tx=0x7f515c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f515bfff640 1 -- 172.21.15.64:0/1491718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0443f0 con 0x7f516415d440 2023-11-24T06:29:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f515bfff640 1 -- 172.21.15.64:0/1491718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c044570 con 0x7f516415d440 2023-11-24T06:29:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f515bfff640 1 -- 172.21.15.64:0/1491718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c044920 con 0x7f516415d440 2023-11-24T06:29:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.176+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51641f3cb0 con 0x7f516415d440 2023-11-24T06:29:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.177+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51640c4ab0 con 0x7f516415d440 2023-11-24T06:29:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.178+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51640c4910 con 0x7f516415d440 2023-11-24T06:29:34.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.179+0000 7f515bfff640 1 -- 172.21.15.64:0/1491718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f515c044a80 con 0x7f516415d440 2023-11-24T06:29:34.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.180+0000 7f515bfff640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f514406ae40 0x7f514406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.180+0000 7f515bfff640 1 -- 172.21.15.64:0/1491718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f515c02d040 con 0x7f516415d440 2023-11-24T06:29:34.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.180+0000 7f5162ffd640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f514406ae40 0x7f514406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.181+0000 7f5162ffd640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f514406ae40 0x7f514406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f51640c6a00 tx=0x7f5154007310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.184+0000 7f515bfff640 1 -- 172.21.15.64:0/1491718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51640c4910 con 0x7f516415d440 2023-11-24T06:29:34.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.418+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51640c52b0 con 0x7f516415d440 2023-11-24T06:29:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.419+0000 7f515bfff640 1 -- 172.21.15.64:0/1491718 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f51640c52b0 con 0x7f516415d440 2023-11-24T06:29:34.423 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:34.423 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3831306721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3831306721},{"type":"v1","addr":"172.21.15.162:6834","nonce":3831306721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1401622125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1401622125},{"type":"v1","addr":"172.21.15.162:6835","nonce":1401622125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2378368513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6835","nonce":2378368513},{"type":"v1","addr":"172.21.15.64:6837","nonce":2378368513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:10.724367+0000","modified":"2023-11-24T06:29:13.855414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6836/2541249514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2541249514},{"type":"v1","addr":"172.21.15.64:6836","nonce":2541249514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:29:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T06:29:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.421+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f514406ae40 msgr2=0x7f514406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.421+0000 7f5163fff640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f514406ae40 0x7f514406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f51640c6a00 tx=0x7f5154007310 comp rx=0 tx=0).stop 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.421+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516415d440 msgr2=0x7f516409cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.421+0000 7f5163fff640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516415d440 0x7f516409cb90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f515c02b950 tx=0x7f515c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 shutdown_connections 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f514406ae40 0x7f514406d300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516415d440 0x7f516409cb90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f516406c220 0x7f516409c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 --2- 172.21.15.64:0/1491718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f516406b700 0x7f5164152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 >> 172.21.15.64:0/1491718 conn(0x7f51640d0de0 msgr2=0x7f51640cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 shutdown_connections 2023-11-24T06:29:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.422+0000 7f5163fff640 1 -- 172.21.15.64:0/1491718 wait complete. 2023-11-24T06:29:34.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:34.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.666+0000 7f0f7b59e640 1 Processor -- start 2023-11-24T06:29:34.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.667+0000 7f0f7b59e640 1 -- start start 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f7b59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f7c0d0100 0x7f0f7c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f7b59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f7c0d0f40 0x7f0f7c0d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f7b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c0d0a10 con 0x7f0f7c160920 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f7b59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7c0cf080 con 0x7f0f7c0d0100 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f7b59e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7c0c6be0 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f7a59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0f7c160920 0x7f0f7c0cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43710/0 (socket says 172.21.15.64:43710) 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f7a59c640 1 -- 172.21.15.64:0/940602386 learned_addr learned my addr 172.21.15.64:0/940602386 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:34.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.668+0000 7f0f79d9b640 1 --2- 172.21.15.64:0/940602386 >> v2:172.21.15.64:3300/0 conn(0x7f0f7c0d0100 0x7f0f7c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.669+0000 7f0f7959a640 1 -- 172.21.15.64:0/940602386 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289245567 0 0) 0x7f0f7c0d0a10 con 0x7f0f7c160920 2023-11-24T06:29:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.669+0000 7f0f7959a640 1 -- 172.21.15.64:0/940602386 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f68003520 con 0x7f0f7c160920 2023-11-24T06:29:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.669+0000 7f0f7ad9d640 1 --2- 172.21.15.64:0/940602386 >> v2:172.21.15.162:3300/0 conn(0x7f0f7c0d0f40 0x7f0f7c0d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.669+0000 7f0f79d9b640 1 -- 172.21.15.64:0/940602386 >> v2:172.21.15.162:3300/0 conn(0x7f0f7c0d0f40 msgr2=0x7f0f7c0d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.669+0000 7f0f79d9b640 1 --2- 172.21.15.64:0/940602386 >> v2:172.21.15.162:3300/0 conn(0x7f0f7c0d0f40 0x7f0f7c0d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.669+0000 7f0f79d9b640 1 -- 172.21.15.64:0/940602386 >> v1:172.21.15.64:6789/0 conn(0x7f0f7c160920 legacy=0x7f0f7c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.669+0000 7f0f79d9b640 1 -- 172.21.15.64:0/940602386 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c0ce7e0 con 0x7f0f7c0d0100 2023-11-24T06:29:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.670+0000 7f0f79d9b640 1 --2- 172.21.15.64:0/940602386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0100 0x7f0f7c0b3f30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c1620b0 tx=0x7f0f6402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fbf9f5adc05e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.670+0000 7f0f7959a640 1 -- 172.21.15.64:0/940602386 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6402cc70 con 0x7f0f7c0d0100 2023-11-24T06:29:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7959a640 1 -- 172.21.15.64:0/940602386 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6402cdd0 con 0x7f0f7c0d0100 2023-11-24T06:29:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7959a640 1 -- 172.21.15.64:0/940602386 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64034640 con 0x7f0f7c0d0100 2023-11-24T06:29:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 -- 172.21.15.64:0/940602386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0100 msgr2=0x7f0f7c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/940602386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0100 0x7f0f7c0b3f30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c1620b0 tx=0x7f0f6402bec0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 -- 172.21.15.64:0/940602386 shutdown_connections 2023-11-24T06:29:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/940602386 >> v2:172.21.15.162:3300/0 conn(0x7f0f7c0d0f40 0x7f0f7c0d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/940602386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0100 0x7f0f7c0b3f30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 -- 172.21.15.64:0/940602386 >> 172.21.15.64:0/940602386 conn(0x7f0f7c0d3090 msgr2=0x7f0f7c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 -- 172.21.15.64:0/940602386 shutdown_connections 2023-11-24T06:29:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.671+0000 7f0f7b59e640 1 -- 172.21.15.64:0/940602386 wait complete. 2023-11-24T06:29:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.673+0000 7f0f7b59e640 1 Processor -- start 2023-11-24T06:29:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.673+0000 7f0f7b59e640 1 -- start start 2023-11-24T06:29:34.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.674+0000 7f0f7b59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c0d0100 0x7f0f7c0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.674+0000 7f0f7b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0f40 0x7f0f7c1fa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.674+0000 7f0f7b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f7c160920 0x7f0f7c1fae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.674+0000 7f0f7b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0b1a80 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.674+0000 7f0f7b59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c1502d0 con 0x7f0f7c0d0100 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.674+0000 7f0f7b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c1505d0 con 0x7f0f7c160920 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.674+0000 7f0f79d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0f40 0x7f0f7c1fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0f40 0x7f0f7c1fa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49690/0 (socket says 172.21.15.64:49690) 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 -- 172.21.15.64:0/3525544869 learned_addr learned my addr 172.21.15.64:0/3525544869 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f7a59c640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c0d0100 0x7f0f7c0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 -- 172.21.15.64:0/3525544869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f7c160920 msgr2=0x7f0f7c1fae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f7c160920 0x7f0f7c1fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 -- 172.21.15.64:0/3525544869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c0d0100 msgr2=0x7f0f7c0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c0d0100 0x7f0f7c0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 -- 172.21.15.64:0/3525544869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64009a70 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.675+0000 7f0f79d9b640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0f40 0x7f0f7c1fa930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009e80 tx=0x7f0f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.676+0000 7f0f72ffd640 1 -- 172.21.15.64:0/3525544869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64007000 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.676+0000 7f0f72ffd640 1 -- 172.21.15.64:0/3525544869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6403e610 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.676+0000 7f0f72ffd640 1 -- 172.21.15.64:0/3525544869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6403e9c0 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.676+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c150890 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.676+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c150e50 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.677+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c067450 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.678+0000 7f0f72ffd640 1 -- 172.21.15.64:0/3525544869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f6403b050 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.679+0000 7f0f72ffd640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f5c06ae40 0x7f0f5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.679+0000 7f0f72ffd640 1 -- 172.21.15.64:0/3525544869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f640abec0 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.679+0000 7f0f7a59c640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f5c06ae40 0x7f0f5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.680+0000 7f0f7a59c640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f5c06ae40 0x7f0f5c06d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005880 tx=0x7f0f6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.684+0000 7f0f72ffd640 1 -- 172.21.15.64:0/3525544869 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f64076f70 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.922+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f7c0b3f30 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.923+0000 7f0f72ffd640 1 -- 172.21.15.64:0/3525544869 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f0f6404e090 con 0x7f0f7c0d0f40 2023-11-24T06:29:34.927 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:34.927 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3831306721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3831306721},{"type":"v1","addr":"172.21.15.162:6834","nonce":3831306721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1401622125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1401622125},{"type":"v1","addr":"172.21.15.162:6835","nonce":1401622125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2378368513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6835","nonce":2378368513},{"type":"v1","addr":"172.21.15.64:6837","nonce":2378368513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:10.724367+0000","modified":"2023-11-24T06:29:13.855414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6836/2541249514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2541249514},{"type":"v1","addr":"172.21.15.64:6836","nonce":2541249514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:29:34.927 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T06:29:34.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f5c06ae40 msgr2=0x7f0f5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f5c06ae40 0x7f0f5c06d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005880 tx=0x7f0f6c009040 comp rx=0 tx=0).stop 2023-11-24T06:29:34.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0f40 msgr2=0x7f0f7c1fa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0f40 0x7f0f7c1fa930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009e80 tx=0x7f0f64004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:34.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 shutdown_connections 2023-11-24T06:29:34.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f0f5c06ae40 0x7f0f5c06d300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f7c160920 0x7f0f7c1fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0d0f40 0x7f0f7c1fa930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 --2- 172.21.15.64:0/3525544869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f7c0d0100 0x7f0f7c0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 >> 172.21.15.64:0/3525544869 conn(0x7f0f7c0d3090 msgr2=0x7f0f7c06aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.926+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 shutdown_connections 2023-11-24T06:29:34.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:34.927+0000 7f0f7b59e640 1 -- 172.21.15.64:0/3525544869 wait complete. 2023-11-24T06:29:34.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:35.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.171+0000 7f3e38168640 1 Processor -- start 2023-11-24T06:29:35.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.171+0000 7f3e38168640 1 -- start start 2023-11-24T06:29:35.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.172+0000 7f3e38168640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3e30143ae0 0x7f3e300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.173+0000 7f3e38168640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e3015e3b0 0x7f3e300702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.173+0000 7f3e38168640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e300cfc30 con 0x7f3e300cf7d0 2023-11-24T06:29:35.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.173+0000 7f3e38168640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e300b1ee0 con 0x7f3e30143ae0 2023-11-24T06:29:35.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.173+0000 7f3e38168640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300b2780 con 0x7f3e3015e3b0 2023-11-24T06:29:35.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.173+0000 7f3e35f04640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3e30143ae0 0x7f3e300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.173+0000 7f3e36705640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e3015e3b0 0x7f3e300702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.174+0000 7f3e35f04640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3e30143ae0 0x7f3e300cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:48286/0 (socket says 172.21.15.64:48286) 2023-11-24T06:29:35.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.174+0000 7f3e35f04640 1 -- 172.21.15.64:0/573616475 learned_addr learned my addr 172.21.15.64:0/573616475 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.174+0000 7f3e36705640 1 -- 172.21.15.64:0/573616475 >> v2:172.21.15.162:3300/0 conn(0x7f3e30143ae0 msgr2=0x7f3e300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.174+0000 7f3e36705640 1 --2- 172.21.15.64:0/573616475 >> v2:172.21.15.162:3300/0 conn(0x7f3e30143ae0 0x7f3e300cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.174+0000 7f3e36705640 1 -- 172.21.15.64:0/573616475 >> v1:172.21.15.162:6789/0 conn(0x7f3e300cf7d0 legacy=0x7f3e3015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.174+0000 7f3e36705640 1 -- 172.21.15.64:0/573616475 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3015e990 con 0x7f3e3015e3b0 2023-11-24T06:29:35.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.174+0000 7f3e35f04640 1 --2- 172.21.15.64:0/573616475 >> v2:172.21.15.162:3300/0 conn(0x7f3e30143ae0 0x7f3e300cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:35.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.176+0000 7f3e36705640 1 --2- 172.21.15.64:0/573616475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e3015e3b0 0x7f3e300702c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e280097a0 tx=0x7f3e2802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79ae2c574d3fad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.176+0000 7f3e34f02640 1 -- 172.21.15.64:0/573616475 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2802d040 con 0x7f3e3015e3b0 2023-11-24T06:29:35.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e34f02640 1 -- 172.21.15.64:0/573616475 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e28031810 con 0x7f3e3015e3b0 2023-11-24T06:29:35.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e34f02640 1 -- 172.21.15.64:0/573616475 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28031ba0 con 0x7f3e3015e3b0 2023-11-24T06:29:35.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 -- 172.21.15.64:0/573616475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e3015e3b0 msgr2=0x7f3e300702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 --2- 172.21.15.64:0/573616475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e3015e3b0 0x7f3e300702c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e280097a0 tx=0x7f3e2802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 -- 172.21.15.64:0/573616475 shutdown_connections 2023-11-24T06:29:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 --2- 172.21.15.64:0/573616475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e3015e3b0 0x7f3e300702c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 --2- 172.21.15.64:0/573616475 >> v2:172.21.15.162:3300/0 conn(0x7f3e30143ae0 0x7f3e300cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 -- 172.21.15.64:0/573616475 >> 172.21.15.64:0/573616475 conn(0x7f3e3006a800 msgr2=0x7f3e3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 -- 172.21.15.64:0/573616475 shutdown_connections 2023-11-24T06:29:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.177+0000 7f3e38168640 1 -- 172.21.15.64:0/573616475 wait complete. 2023-11-24T06:29:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.179+0000 7f3e38168640 1 Processor -- start 2023-11-24T06:29:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.179+0000 7f3e38168640 1 -- start start 2023-11-24T06:29:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.180+0000 7f3e38168640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e300cf7d0 0x7f3e30141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.180+0000 7f3e38168640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e30143ae0 0x7f3e300aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.180+0000 7f3e38168640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e3015e3b0 0x7f3e3013d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.180+0000 7f3e38168640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300aff80 con 0x7f3e3015e3b0 2023-11-24T06:29:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.180+0000 7f3e38168640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e301eee30 con 0x7f3e300cf7d0 2023-11-24T06:29:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.180+0000 7f3e38168640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300af400 con 0x7f3e30143ae0 2023-11-24T06:29:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e3015e3b0 0x7f3e3013d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e3015e3b0 0x7f3e3013d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49694/0 (socket says 172.21.15.64:49694) 2023-11-24T06:29:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 -- 172.21.15.64:0/304810092 learned_addr learned my addr 172.21.15.64:0/304810092 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:35.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e35f04640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e300cf7d0 0x7f3e30141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e35703640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e30143ae0 0x7f3e300aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 -- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e30143ae0 msgr2=0x7f3e300aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e30143ae0 0x7f3e300aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 -- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e300cf7d0 msgr2=0x7f3e30141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e300cf7d0 0x7f3e30141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 -- 172.21.15.64:0/304810092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2400bd40 con 0x7f3e3015e3b0 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e35703640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e30143ae0 0x7f3e300aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.181+0000 7f3e36705640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e3015e3b0 0x7f3e3013d4c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3e280097a0 tx=0x7f3e280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.182+0000 7f3e2e7fc640 1 -- 172.21.15.64:0/304810092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28006e10 con 0x7f3e3015e3b0 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.182+0000 7f3e2e7fc640 1 -- 172.21.15.64:0/304810092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2803ace0 con 0x7f3e3015e3b0 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.182+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e28009400 con 0x7f3e3015e3b0 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.182+0000 7f3e2e7fc640 1 -- 172.21.15.64:0/304810092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28044610 con 0x7f3e3015e3b0 2023-11-24T06:29:35.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.182+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e300af9c0 con 0x7f3e3015e3b0 2023-11-24T06:29:35.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.183+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e300675d0 con 0x7f3e3015e3b0 2023-11-24T06:29:35.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.184+0000 7f3e2e7fc640 1 -- 172.21.15.64:0/304810092 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e2804d450 con 0x7f3e3015e3b0 2023-11-24T06:29:35.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.185+0000 7f3e2e7fc640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3e1406ae40 0x7f3e1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.185+0000 7f3e2e7fc640 1 -- 172.21.15.64:0/304810092 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e280ab780 con 0x7f3e3015e3b0 2023-11-24T06:29:35.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.185+0000 7f3e35f04640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3e1406ae40 0x7f3e1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.186+0000 7f3e35f04640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3e1406ae40 0x7f3e1406d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400bfd0 tx=0x7f3e2400d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.190+0000 7f3e2e7fc640 1 -- 172.21.15.64:0/304810092 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e28076820 con 0x7f3e3015e3b0 2023-11-24T06:29:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.392+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e301eeff0 con 0x7f3e3015e3b0 2023-11-24T06:29:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.394+0000 7f3e2e7fc640 1 -- 172.21.15.64:0/304810092 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f3e28079e40 con 0x7f3e3015e3b0 2023-11-24T06:29:35.398 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:35.399 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:12.271627+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:08.540135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:09.752538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3e1406ae40 msgr2=0x7f3e1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3e1406ae40 0x7f3e1406d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400bfd0 tx=0x7f3e2400d040 comp rx=0 tx=0).stop 2023-11-24T06:29:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e3015e3b0 msgr2=0x7f3e3013d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e3015e3b0 0x7f3e3013d4c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3e280097a0 tx=0x7f3e280046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 shutdown_connections 2023-11-24T06:29:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f3e1406ae40 0x7f3e1406d300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e3015e3b0 0x7f3e3013d4c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e30143ae0 0x7f3e300aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 --2- 172.21.15.64:0/304810092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e300cf7d0 0x7f3e30141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 >> 172.21.15.64:0/304810092 conn(0x7f3e3006a800 msgr2=0x7f3e30142af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.397+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 shutdown_connections 2023-11-24T06:29:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.398+0000 7f3e38168640 1 -- 172.21.15.64:0/304810092 wait complete. 2023-11-24T06:29:35.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:29:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.647+0000 7f1ec1383640 1 Processor -- start 2023-11-24T06:29:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.647+0000 7f1ec1383640 1 -- start start 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ec1383640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc15afb0 0x7f1ebc13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ec1383640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ebc13fae0 0x7f1ebc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ec1383640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc13ff00 con 0x7f1ebc13cfa0 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ec1383640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0cdc40 con 0x7f1ebc13fae0 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ec1383640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0cdec0 con 0x7f1ebc15afb0 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ebbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1ebc13cfa0 0x7f1ebc0cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43716/0 (socket says 172.21.15.64:43716) 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ebbfff640 1 -- 172.21.15.64:0/3174660255 learned_addr learned my addr 172.21.15.64:0/3174660255 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1ec0b82640 1 --2- 172.21.15.64:0/3174660255 >> v2:172.21.15.64:3300/0 conn(0x7f1ebc13fae0 0x7f1ebc0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.649+0000 7f1eb37fe640 1 --2- 172.21.15.64:0/3174660255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc15afb0 0x7f1ebc13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.650+0000 7f1ebb7fe640 1 -- 172.21.15.64:0/3174660255 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692032173 0 0) 0x7f1ebc13ff00 con 0x7f1ebc13cfa0 2023-11-24T06:29:35.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.650+0000 7f1ebb7fe640 1 -- 172.21.15.64:0/3174660255 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea8005730 con 0x7f1ebc13cfa0 2023-11-24T06:29:35.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.650+0000 7f1ec0b82640 1 -- 172.21.15.64:0/3174660255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc15afb0 msgr2=0x7f1ebc13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:35.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.650+0000 7f1ec0b82640 1 --2- 172.21.15.64:0/3174660255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc15afb0 0x7f1ebc13f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.650+0000 7f1ec0b82640 1 -- 172.21.15.64:0/3174660255 >> v1:172.21.15.64:6789/0 conn(0x7f1ebc13cfa0 legacy=0x7f1ebc0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.650+0000 7f1ec0b82640 1 -- 172.21.15.64:0/3174660255 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0ce1a0 con 0x7f1ebc13fae0 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.651+0000 7f1ec0b82640 1 --2- 172.21.15.64:0/3174660255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc13fae0 0x7f1ebc0b2040 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40097a0 tx=0x7f1eb402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df26f71971ca1dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.651+0000 7f1ebb7fe640 1 -- 172.21.15.64:0/3174660255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb402e070 con 0x7f1ebc13fae0 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.651+0000 7f1ebb7fe640 1 -- 172.21.15.64:0/3174660255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40315a0 con 0x7f1ebc13fae0 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.651+0000 7f1ebb7fe640 1 -- 172.21.15.64:0/3174660255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4031930 con 0x7f1ebc13fae0 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 -- 172.21.15.64:0/3174660255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc13fae0 msgr2=0x7f1ebc0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3174660255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc13fae0 0x7f1ebc0b2040 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40097a0 tx=0x7f1eb402b840 comp rx=0 tx=0).stop 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 -- 172.21.15.64:0/3174660255 shutdown_connections 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3174660255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc13fae0 0x7f1ebc0b2040 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3174660255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc15afb0 0x7f1ebc13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 -- 172.21.15.64:0/3174660255 >> 172.21.15.64:0/3174660255 conn(0x7f1ebc06a800 msgr2=0x7f1ebc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 -- 172.21.15.64:0/3174660255 shutdown_connections 2023-11-24T06:29:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.652+0000 7f1ec1383640 1 -- 172.21.15.64:0/3174660255 wait complete. 2023-11-24T06:29:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.654+0000 7f1ec1383640 1 Processor -- start 2023-11-24T06:29:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.654+0000 7f1ec1383640 1 -- start start 2023-11-24T06:29:35.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec1383640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ebc13cfa0 0x7f1ebc138410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec1383640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc13fae0 0x7f1ebc13e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec1383640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc15afb0 0x7f1ebc13e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec1383640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc13aa80 con 0x7f1ebc15afb0 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec1383640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc13ad60 con 0x7f1ebc13cfa0 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec1383640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc13b040 con 0x7f1ebc13fae0 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec0b82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc15afb0 0x7f1ebc13e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec0b82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc15afb0 0x7f1ebc13e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49724/0 (socket says 172.21.15.64:49724) 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.655+0000 7f1ec0b82640 1 -- 172.21.15.64:0/3280029594 learned_addr learned my addr 172.21.15.64:0/3280029594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1eb37fe640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc13fae0 0x7f1ebc13e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ebbfff640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ebc13cfa0 0x7f1ebc138410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ec0b82640 1 -- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc13fae0 msgr2=0x7f1ebc13e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ec0b82640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc13fae0 0x7f1ebc13e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ec0b82640 1 -- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ebc13cfa0 msgr2=0x7f1ebc138410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ec0b82640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ebc13cfa0 0x7f1ebc138410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ec0b82640 1 -- 172.21.15.64:0/3280029594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4009400 con 0x7f1ebc15afb0 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1eb37fe640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc13fae0 0x7f1ebc13e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ebbfff640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ebc13cfa0 0x7f1ebc138410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.656+0000 7f1ec0b82640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc15afb0 0x7f1ebc13e9a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034040 tx=0x7f1eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.657+0000 7f1eb97fa640 1 -- 172.21.15.64:0/3280029594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4032840 con 0x7f1ebc15afb0 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.657+0000 7f1eb97fa640 1 -- 172.21.15.64:0/3280029594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40329c0 con 0x7f1ebc15afb0 2023-11-24T06:29:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.657+0000 7f1eb97fa640 1 -- 172.21.15.64:0/3280029594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4032d70 con 0x7f1ebc15afb0 2023-11-24T06:29:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.657+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc151c90 con 0x7f1ebc15afb0 2023-11-24T06:29:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.657+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc138b30 con 0x7f1ebc15afb0 2023-11-24T06:29:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.658+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc151af0 con 0x7f1ebc15afb0 2023-11-24T06:29:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.659+0000 7f1eb97fa640 1 -- 172.21.15.64:0/3280029594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1eb403b3f0 con 0x7f1ebc15afb0 2023-11-24T06:29:35.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.660+0000 7f1eb97fa640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1e9c06ae40 0x7f1e9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.661+0000 7f1eb97fa640 1 -- 172.21.15.64:0/3280029594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eb40ab3a0 con 0x7f1ebc15afb0 2023-11-24T06:29:35.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.661+0000 7f1ebbfff640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1e9c06ae40 0x7f1e9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.661+0000 7f1ebbfff640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1e9c06ae40 0x7f1e9c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1eac001560 tx=0x7f1eac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.665+0000 7f1eb97fa640 1 -- 172.21.15.64:0/3280029594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ebc151af0 con 0x7f1ebc15afb0 2023-11-24T06:29:35.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.898+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1ebc138b30 con 0x7f1ebc15afb0 2023-11-24T06:29:35.903 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T06:29:35.899+0000 7fa0b5f4b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:29:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.903+0000 7f1eb97fa640 1 -- 172.21.15.64:0/3280029594 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1ebc138b30 con 0x7f1ebc15afb0 2023-11-24T06:29:35.908 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:29:35.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.907+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1e9c06ae40 msgr2=0x7f1e9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.907+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1e9c06ae40 0x7f1e9c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1eac001560 tx=0x7f1eac009040 comp rx=0 tx=0).stop 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.907+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc15afb0 msgr2=0x7f1ebc13e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.907+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc15afb0 0x7f1ebc13e9a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034040 tx=0x7f1eb40046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 shutdown_connections 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f1e9c06ae40 0x7f1e9c06d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc15afb0 0x7f1ebc13e9a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ebc13fae0 0x7f1ebc13e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 --2- 172.21.15.64:0/3280029594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ebc13cfa0 0x7f1ebc138410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 >> 172.21.15.64:0/3280029594 conn(0x7f1ebc06a800 msgr2=0x7f1ebc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 shutdown_connections 2023-11-24T06:29:35.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:35.908+0000 7f1ec1383640 1 -- 172.21.15.64:0/3280029594 wait complete. 2023-11-24T06:29:35.917 INFO:tasks.ceph.mds.a.smithi064.stderr: -1210> 2023-11-24T06:29:12.209+0000 7f7f598e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:29:35.931 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:29:35.976 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T06:29:36.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.122+0000 7f903b59e640 1 Processor -- start 2023-11-24T06:29:36.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.122+0000 7f903b59e640 1 -- start start 2023-11-24T06:29:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.124+0000 7f903b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 0x7f903c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.124+0000 7f903b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c15f4e0 con 0x7f903c15bf10 2023-11-24T06:29:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.124+0000 7f903b59e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c071130 con 0x7f903c0ccb50 2023-11-24T06:29:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.124+0000 7f903b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c15f970 con 0x7f903c0adf20 2023-11-24T06:29:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.125+0000 7f9033fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f903c15bf10 0x7f903c0cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43726/0 (socket says 172.21.15.64:43726) 2023-11-24T06:29:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.125+0000 7f9033fff640 1 -- 172.21.15.64:0/3070919006 learned_addr learned my addr 172.21.15.64:0/3070919006 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:36.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.125+0000 7f903a59c640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 0x7f903c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.125+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137885291 0 0) 0x7f903c15f4e0 con 0x7f903c15bf10 2023-11-24T06:29:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.125+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9028003520 con 0x7f903c15bf10 2023-11-24T06:29:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1558899275 0 0) 0x7f9028003520 con 0x7f903c15bf10 2023-11-24T06:29:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 msgr2=0x7f903c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 0x7f903c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 >> v1:172.21.15.162:6789/0 conn(0x7f903c0ccb50 legacy=0x7f903c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c15c390 con 0x7f903c15bf10 2023-11-24T06:29:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f9024001670 con 0x7f903c15bf10 2023-11-24T06:29:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 >> v1:172.21.15.64:6789/0 conn(0x7f903c15bf10 legacy=0x7f903c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 0x7f90280039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9028003f70 0x7f90280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028004d00 0x7f9028005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c15f970 con 0x7f9028003f70 2023-11-24T06:29:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902800bb70 con 0x7f9028004d00 2023-11-24T06:29:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902800bd30 con 0x7f903c0adf20 2023-11-24T06:29:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f9033fff640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9028003f70 0x7f90280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f903ad9d640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028004d00 0x7f9028005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.126+0000 7f903a59c640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 0x7f90280039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.127+0000 7f9033fff640 1 -- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 msgr2=0x7f90280039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.127+0000 7f9033fff640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0adf20 0x7f90280039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.127+0000 7f9033fff640 1 -- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028004d00 msgr2=0x7f9028005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.127+0000 7f9033fff640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028004d00 0x7f9028005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.127+0000 7f9033fff640 1 -- 172.21.15.64:0/3070919006 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028003520 con 0x7f9028003f70 2023-11-24T06:29:36.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.127+0000 7f903ad9d640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028004d00 0x7f9028005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:36.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.128+0000 7f9033fff640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9028003f70 0x7f90280047c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f902400c0e0 tx=0x7f90280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decb2869dc6b61a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.128+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024005ad0 con 0x7f9028003f70 2023-11-24T06:29:36.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.128+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9024005c50 con 0x7f9028003f70 2023-11-24T06:29:36.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.128+0000 7f9039d9b640 1 -- 172.21.15.64:0/3070919006 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902402c7d0 con 0x7f9028003f70 2023-11-24T06:29:36.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.129+0000 7f903b59e640 1 -- 172.21.15.64:0/3070919006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9028003f70 msgr2=0x7f90280047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.129+0000 7f903b59e640 1 --2- 172.21.15.64:0/3070919006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9028003f70 0x7f90280047c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f902400c0e0 tx=0x7f90280057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.129+0000 7f903ad9d640 1 -- 172.21.15.64:0/3070919006 reap_dead start 2023-11-24T06:29:36.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.129+0000 7f903b59e640 1 -- 172.21.15.64:0/3070919006 shutdown_connections 2023-11-24T06:29:36.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.129+0000 7f903b59e640 1 -- 172.21.15.64:0/3070919006 >> 172.21.15.64:0/3070919006 conn(0x7f903c06a7d0 msgr2=0x7f903c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:36.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.129+0000 7f903b59e640 1 -- 172.21.15.64:0/3070919006 shutdown_connections 2023-11-24T06:29:36.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.129+0000 7f903b59e640 1 -- 172.21.15.64:0/3070919006 wait complete. 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.131+0000 7f903b59e640 1 Processor -- start 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.131+0000 7f903b59e640 1 -- start start 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903b59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028003f70 0x7f903c0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903b59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0b2850 0x7f903c13f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c15db10 0x7f903c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c15e4d0 con 0x7f903c15db10 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903b59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c0caef0 con 0x7f9028003f70 2023-11-24T06:29:36.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903b59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c13ee30 con 0x7f903c0b2850 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c15db10 0x7f903c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c15db10 0x7f903c15df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49760/0 (socket says 172.21.15.64:49760) 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.132+0000 7f903ad9d640 1 -- 172.21.15.64:0/888573019 learned_addr learned my addr 172.21.15.64:0/888573019 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903a59c640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028003f70 0x7f903c0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f9033fff640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0b2850 0x7f903c13f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903ad9d640 1 -- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0b2850 msgr2=0x7f903c13f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903ad9d640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0b2850 0x7f903c13f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903ad9d640 1 -- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028003f70 msgr2=0x7f903c0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903ad9d640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028003f70 0x7f903c0ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903ad9d640 1 -- 172.21.15.64:0/888573019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c13f790 con 0x7f903c15db10 2023-11-24T06:29:36.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f9033fff640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0b2850 0x7f903c13f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:36.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903a59c640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028003f70 0x7f903c0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:36.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903ad9d640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c15db10 0x7f903c15df90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f903c15fb10 tx=0x7f903400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f9032ffd640 1 -- 172.21.15.64:0/888573019 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034006d50 con 0x7f903c15db10 2023-11-24T06:29:36.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f9032ffd640 1 -- 172.21.15.64:0/888573019 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9034003970 con 0x7f903c15db10 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f9032ffd640 1 -- 172.21.15.64:0/888573019 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034003d60 con 0x7f903c15db10 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.133+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c0b2d80 con 0x7f903c15db10 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.134+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c141f50 con 0x7f903c15db10 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.134+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903c067450 con 0x7f903c15db10 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.136+0000 7f9032ffd640 1 -- 172.21.15.64:0/888573019 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9034005070 con 0x7f903c15db10 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.136+0000 7f9032ffd640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f901c06ae70 0x7f901c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.137+0000 7f9032ffd640 1 -- 172.21.15.64:0/888573019 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9034089690 con 0x7f903c15db10 2023-11-24T06:29:36.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.137+0000 7f903a59c640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f901c06ae70 0x7f901c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.137+0000 7f903a59c640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f901c06ae70 0x7f901c06d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f902c0099e0 tx=0x7f902c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.141+0000 7f9032ffd640 1 -- 172.21.15.64:0/888573019 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90340545b0 con 0x7f903c15db10 2023-11-24T06:29:36.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.375+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f903c15bcd0 con 0x7f903c15db10 2023-11-24T06:29:36.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.941+0000 7f9032ffd640 1 -- 172.21.15.64:0/888573019 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9034057bd0 con 0x7f903c15db10 2023-11-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f901c06ae70 msgr2=0x7f901c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f901c06ae70 0x7f901c06d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f902c0099e0 tx=0x7f902c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c15db10 msgr2=0x7f903c15df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c15db10 0x7f903c15df90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f903c15fb10 tx=0x7f903400dc00 comp rx=0 tx=0).stop 2023-11-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 shutdown_connections 2023-11-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f901c06ae70 0x7f901c06d330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c15db10 0x7f903c15df90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f903c0b2850 0x7f903c13f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 --2- 172.21.15.64:0/888573019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9028003f70 0x7f903c0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 >> 172.21.15.64:0/888573019 conn(0x7f903c06a7d0 msgr2=0x7f903c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 shutdown_connections 2023-11-24T06:29:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:36.945+0000 7f903b59e640 1 -- 172.21.15.64:0/888573019 wait complete. 2023-11-24T06:29:36.974 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:29:37.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.246+0000 7fe8ee92c640 1 Processor -- start 2023-11-24T06:29:37.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.246+0000 7fe8ee92c640 1 -- start start 2023-11-24T06:29:37.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8ee92c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e806a3e0 0x7fe8e80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8ee92c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8e80aede0 0x7fe8e806f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8ee92c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e8071de0 con 0x7fe8e80d0b60 2023-11-24T06:29:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8ee92c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e80af270 con 0x7fe8e80aede0 2023-11-24T06:29:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8ee92c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8072030 con 0x7fe8e806a3e0 2023-11-24T06:29:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8e77fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe8e80d0b60 0x7fe8e806a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43738/0 (socket says 172.21.15.64:43738) 2023-11-24T06:29:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8e77fe640 1 -- 172.21.15.64:0/1707714751 learned_addr learned my addr 172.21.15.64:0/1707714751 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8ecec9640 1 --2- 172.21.15.64:0/1707714751 >> v2:172.21.15.64:3300/0 conn(0x7fe8e80aede0 0x7fe8e806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.248+0000 7fe8e7fff640 1 --2- 172.21.15.64:0/1707714751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e806a3e0 0x7fe8e80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.249+0000 7fe8e6ffd640 1 -- 172.21.15.64:0/1707714751 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503957147 0 0) 0x7fe8e8071de0 con 0x7fe8e80d0b60 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.249+0000 7fe8e6ffd640 1 -- 172.21.15.64:0/1707714751 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d0003520 con 0x7fe8e80d0b60 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.249+0000 7fe8ecec9640 1 -- 172.21.15.64:0/1707714751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e806a3e0 msgr2=0x7fe8e80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.249+0000 7fe8ecec9640 1 --2- 172.21.15.64:0/1707714751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e806a3e0 0x7fe8e80ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.249+0000 7fe8ecec9640 1 -- 172.21.15.64:0/1707714751 >> v1:172.21.15.64:6789/0 conn(0x7fe8e80d0b60 legacy=0x7fe8e806a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.249+0000 7fe8ecec9640 1 -- 172.21.15.64:0/1707714751 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e8072310 con 0x7fe8e80aede0 2023-11-24T06:29:37.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.249+0000 7fe8e7fff640 1 --2- 172.21.15.64:0/1707714751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e806a3e0 0x7fe8e80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.250+0000 7fe8ecec9640 1 --2- 172.21.15.64:0/1707714751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e80aede0 0x7fe8e806f080 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009d90 tx=0x7fe8e002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7e13d90c9c2154 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.250+0000 7fe8e6ffd640 1 -- 172.21.15.64:0/1707714751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e002cc50 con 0x7fe8e80aede0 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.250+0000 7fe8e6ffd640 1 -- 172.21.15.64:0/1707714751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e002cdb0 con 0x7fe8e80aede0 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.250+0000 7fe8e6ffd640 1 -- 172.21.15.64:0/1707714751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0034660 con 0x7fe8e80aede0 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 -- 172.21.15.64:0/1707714751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e80aede0 msgr2=0x7fe8e806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/1707714751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e80aede0 0x7fe8e806f080 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009d90 tx=0x7fe8e002bea0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 -- 172.21.15.64:0/1707714751 shutdown_connections 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/1707714751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e80aede0 0x7fe8e806f080 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/1707714751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e806a3e0 0x7fe8e80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 -- 172.21.15.64:0/1707714751 >> 172.21.15.64:0/1707714751 conn(0x7fe8e80cfef0 msgr2=0x7fe8e80d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:37.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 -- 172.21.15.64:0/1707714751 shutdown_connections 2023-11-24T06:29:37.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.251+0000 7fe8ee92c640 1 -- 172.21.15.64:0/1707714751 wait complete. 2023-11-24T06:29:37.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.254+0000 7fe8ee92c640 1 Processor -- start 2023-11-24T06:29:37.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.254+0000 7fe8ee92c640 1 -- start start 2023-11-24T06:29:37.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ee92c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e806a3e0 0x7fe8e80cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ee92c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e80aede0 0x7fe8e80c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ee92c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e80d0b60 0x7fe8e80cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ee92c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80cd5f0 con 0x7fe8e806a3e0 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ee92c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80cc9e0 con 0x7fe8e80aede0 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ee92c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80cccc0 con 0x7fe8e80d0b60 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e806a3e0 0x7fe8e80cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e806a3e0 0x7fe8e80cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49790/0 (socket says 172.21.15.64:49790) 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 -- 172.21.15.64:0/815506865 learned_addr learned my addr 172.21.15.64:0/815506865 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ecec9640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e80d0b60 0x7fe8e80cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 -- 172.21.15.64:0/815506865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e80d0b60 msgr2=0x7fe8e80cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e80d0b60 0x7fe8e80cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 -- 172.21.15.64:0/815506865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e80aede0 msgr2=0x7fe8e80c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e80aede0 0x7fe8e80c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 -- 172.21.15.64:0/815506865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0009a70 con 0x7fe8e806a3e0 2023-11-24T06:29:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e7fff640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e806a3e0 0x7fe8e80cf1c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800d7d0 tx=0x7fe8d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e4ff9640 1 -- 172.21.15.64:0/815506865 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8004030 con 0x7fe8e806a3e0 2023-11-24T06:29:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e4ff9640 1 -- 172.21.15.64:0/815506865 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d8004190 con 0x7fe8e806a3e0 2023-11-24T06:29:37.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8e4ff9640 1 -- 172.21.15.64:0/815506865 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8004520 con 0x7fe8e806a3e0 2023-11-24T06:29:37.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.257+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e80ccfa0 con 0x7fe8e806a3e0 2023-11-24T06:29:37.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.258+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e80cf700 con 0x7fe8e806a3e0 2023-11-24T06:29:37.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.259+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e80675d0 con 0x7fe8e806a3e0 2023-11-24T06:29:37.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.261+0000 7fe8e4ff9640 1 -- 172.21.15.64:0/815506865 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8d800b6c0 con 0x7fe8e806a3e0 2023-11-24T06:29:37.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.263+0000 7fe8e4ff9640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe8d406ae70 0x7fe8d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.263+0000 7fe8e4ff9640 1 -- 172.21.15.64:0/815506865 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe8d8089850 con 0x7fe8e806a3e0 2023-11-24T06:29:37.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.263+0000 7fe8e77fe640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe8d406ae70 0x7fe8d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.264+0000 7fe8e77fe640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe8d406ae70 0x7fe8d406d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80cc1d0 tx=0x7fe8dc007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.269+0000 7fe8e4ff9640 1 -- 172.21.15.64:0/815506865 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8d8054800 con 0x7fe8e806a3e0 2023-11-24T06:29:37.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.466+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe8e815a690 con 0x7fe8e806a3e0 2023-11-24T06:29:37.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.961+0000 7fe8e4ff9640 1 -- 172.21.15.64:0/815506865 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe8d8057e20 con 0x7fe8e806a3e0 2023-11-24T06:29:37.965 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:29:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe8d406ae70 msgr2=0x7fe8d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe8d406ae70 0x7fe8d406d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80cc1d0 tx=0x7fe8dc007930 comp rx=0 tx=0).stop 2023-11-24T06:29:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e806a3e0 msgr2=0x7fe8e80cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e806a3e0 0x7fe8e80cf1c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800d7d0 tx=0x7fe8d800dc90 comp rx=0 tx=0).stop 2023-11-24T06:29:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 shutdown_connections 2023-11-24T06:29:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fe8d406ae70 0x7fe8d406d330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8e80d0b60 0x7fe8e80cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e80aede0 0x7fe8e80c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 --2- 172.21.15.64:0/815506865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e806a3e0 0x7fe8e80cf1c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 >> 172.21.15.64:0/815506865 conn(0x7fe8e80cfef0 msgr2=0x7fe8e80b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:37.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 shutdown_connections 2023-11-24T06:29:37.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:37.965+0000 7fe8ee92c640 1 -- 172.21.15.64:0/815506865 wait complete. 2023-11-24T06:29:37.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:29:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.196+0000 7fb6672bb640 1 Processor -- start 2023-11-24T06:29:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.196+0000 7fb6672bb640 1 -- start start 2023-11-24T06:29:38.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.198+0000 7fb6672bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6601616b0 0x7fb660165c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.198+0000 7fb6672bb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb66006aaf0 0x7fb6600b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.198+0000 7fb6672bb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66006f7c0 con 0x7fb6600b5030 2023-11-24T06:29:38.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.198+0000 7fb6672bb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66006faa0 con 0x7fb66006aaf0 2023-11-24T06:29:38.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.198+0000 7fb6672bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb660165130 con 0x7fb6601616b0 2023-11-24T06:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.199+0000 7fb666aba640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb66006aaf0 0x7fb6600b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.199+0000 7fb6662b9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6601616b0 0x7fb660165c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.199+0000 7fb666aba640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb66006aaf0 0x7fb6600b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:48352/0 (socket says 172.21.15.64:48352) 2023-11-24T06:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.199+0000 7fb666aba640 1 -- 172.21.15.64:0/4270641607 learned_addr learned my addr 172.21.15.64:0/4270641607 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.200+0000 7fb666aba640 1 -- 172.21.15.64:0/4270641607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6601616b0 msgr2=0x7fb660165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.200+0000 7fb666aba640 1 --2- 172.21.15.64:0/4270641607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6601616b0 0x7fb660165c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.200+0000 7fb666aba640 1 -- 172.21.15.64:0/4270641607 >> v1:172.21.15.162:6789/0 conn(0x7fb6600b5030 legacy=0x7fb66006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.200+0000 7fb666aba640 1 -- 172.21.15.64:0/4270641607 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660165410 con 0x7fb66006aaf0 2023-11-24T06:29:38.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.201+0000 7fb666aba640 1 --2- 172.21.15.64:0/4270641607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb66006aaf0 0x7fb6600b39b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb65802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b85ef7766417f610 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.202+0000 7fb6652b7640 1 -- 172.21.15.64:0/4270641607 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65802cb00 con 0x7fb66006aaf0 2023-11-24T06:29:38.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.202+0000 7fb6652b7640 1 -- 172.21.15.64:0/4270641607 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65802cc60 con 0x7fb66006aaf0 2023-11-24T06:29:38.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.202+0000 7fb6652b7640 1 -- 172.21.15.64:0/4270641607 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6580343f0 con 0x7fb66006aaf0 2023-11-24T06:29:38.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.202+0000 7fb6672bb640 1 -- 172.21.15.64:0/4270641607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb66006aaf0 msgr2=0x7fb6600b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.202+0000 7fb6672bb640 1 --2- 172.21.15.64:0/4270641607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb66006aaf0 0x7fb6600b39b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb65802bf60 comp rx=0 tx=0).stop 2023-11-24T06:29:38.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.202+0000 7fb6672bb640 1 -- 172.21.15.64:0/4270641607 shutdown_connections 2023-11-24T06:29:38.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.202+0000 7fb6672bb640 1 --2- 172.21.15.64:0/4270641607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb66006aaf0 0x7fb6600b39b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.203+0000 7fb6672bb640 1 --2- 172.21.15.64:0/4270641607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6601616b0 0x7fb660165c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.203+0000 7fb6672bb640 1 -- 172.21.15.64:0/4270641607 >> 172.21.15.64:0/4270641607 conn(0x7fb6600b1a50 msgr2=0x7fb6600b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:38.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.203+0000 7fb6672bb640 1 -- 172.21.15.64:0/4270641607 shutdown_connections 2023-11-24T06:29:38.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.203+0000 7fb6672bb640 1 -- 172.21.15.64:0/4270641607 wait complete. 2023-11-24T06:29:38.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.204+0000 7fb6672bb640 1 Processor -- start 2023-11-24T06:29:38.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.205+0000 7fb6672bb640 1 -- start start 2023-11-24T06:29:38.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.205+0000 7fb6672bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb66006aaf0 0x7fb66014b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.205+0000 7fb6672bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6600b5030 0x7fb660149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb6672bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6601616b0 0x7fb66009d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb6672bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66014b290 con 0x7fb6601616b0 2023-11-24T06:29:38.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb6672bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66014bc60 con 0x7fb6600b5030 2023-11-24T06:29:38.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb6672bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66014a680 con 0x7fb66006aaf0 2023-11-24T06:29:38.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6601616b0 0x7fb66009d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6601616b0 0x7fb66009d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49794/0 (socket says 172.21.15.64:49794) 2023-11-24T06:29:38.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 -- 172.21.15.64:0/316780072 learned_addr learned my addr 172.21.15.64:0/316780072 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:38.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb6662b9640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb66006aaf0 0x7fb66014b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb665ab8640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6600b5030 0x7fb660149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 -- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb66006aaf0 msgr2=0x7fb66014b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb66006aaf0 0x7fb66014b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 -- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6600b5030 msgr2=0x7fb660149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6600b5030 0x7fb660149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb666aba640 1 -- 172.21.15.64:0/316780072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb658009b30 con 0x7fb6601616b0 2023-11-24T06:29:38.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.206+0000 7fb665ab8640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6600b5030 0x7fb660149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:38.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.207+0000 7fb666aba640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6601616b0 0x7fb66009d910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.207+0000 7fb652ffd640 1 -- 172.21.15.64:0/316780072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658009510 con 0x7fb6601616b0 2023-11-24T06:29:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.207+0000 7fb652ffd640 1 -- 172.21.15.64:0/316780072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65803e450 con 0x7fb6601616b0 2023-11-24T06:29:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.207+0000 7fb652ffd640 1 -- 172.21.15.64:0/316780072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65803e800 con 0x7fb6601616b0 2023-11-24T06:29:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.207+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66014a960 con 0x7fb6601616b0 2023-11-24T06:29:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.207+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66014af20 con 0x7fb6601616b0 2023-11-24T06:29:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.208+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66014b0e0 con 0x7fb6601616b0 2023-11-24T06:29:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.209+0000 7fb652ffd640 1 -- 172.21.15.64:0/316780072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb65803e960 con 0x7fb6601616b0 2023-11-24T06:29:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.210+0000 7fb652ffd640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb64406ae70 0x7fb64406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.210+0000 7fb652ffd640 1 -- 172.21.15.64:0/316780072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fb6580ac170 con 0x7fb6601616b0 2023-11-24T06:29:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.211+0000 7fb6662b9640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb64406ae70 0x7fb64406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.211+0000 7fb6662b9640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb64406ae70 0x7fb64406d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009970 tx=0x7fb65c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.215+0000 7fb652ffd640 1 -- 172.21.15.64:0/316780072 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb658077370 con 0x7fb6601616b0 2023-11-24T06:29:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.415+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb660074a00 con 0x7fb6601616b0 2023-11-24T06:29:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.973+0000 7fb652ffd640 1 -- 172.21.15.64:0/316780072 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb65807a990 con 0x7fb6601616b0 2023-11-24T06:29:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.976+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb64406ae70 msgr2=0x7fb64406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.976+0000 7fb6672bb640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb64406ae70 0x7fb64406d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009970 tx=0x7fb65c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.976+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6601616b0 msgr2=0x7fb66009d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.976+0000 7fb6672bb640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6601616b0 0x7fb66009d910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb658004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 shutdown_connections 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb64406ae70 0x7fb64406d330 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6601616b0 0x7fb66009d910 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6600b5030 0x7fb660149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 --2- 172.21.15.64:0/316780072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb66006aaf0 0x7fb66014b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 >> 172.21.15.64:0/316780072 conn(0x7fb6600b1a50 msgr2=0x7fb66006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:38.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 shutdown_connections 2023-11-24T06:29:38.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:38.977+0000 7fb6672bb640 1 -- 172.21.15.64:0/316780072 wait complete. 2023-11-24T06:29:39.000 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:29:39.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.226+0000 7f5ebe6c9640 1 Processor -- start 2023-11-24T06:29:39.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.226+0000 7f5ebe6c9640 1 -- start start 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5ebe6c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5eb80ae5a0 0x7f5eb815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5ebe6c9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5eb80b3ae0 0x7f5eb806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5ebe6c9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb8070c40 con 0x7f5eb806d120 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5ebe6c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb8071450 con 0x7f5eb80ae5a0 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5ebe6c9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb806c470 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5eb7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5eb80ae5a0 0x7f5eb815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5eb7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5eb80ae5a0 0x7f5eb815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49806/0 (socket says 172.21.15.64:49806) 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5eb7fff640 1 -- 172.21.15.64:0/4265305963 learned_addr learned my addr 172.21.15.64:0/4265305963 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5eb7fff640 1 -- 172.21.15.64:0/4265305963 >> v2:172.21.15.162:3300/0 conn(0x7f5eb80b3ae0 msgr2=0x7f5eb806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:39.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5eb7fff640 1 --2- 172.21.15.64:0/4265305963 >> v2:172.21.15.162:3300/0 conn(0x7f5eb80b3ae0 0x7f5eb806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5eb7fff640 1 -- 172.21.15.64:0/4265305963 >> v1:172.21.15.162:6789/0 conn(0x7f5eb806d120 legacy=0x7f5eb80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:39.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.228+0000 7f5eb7fff640 1 -- 172.21.15.64:0/4265305963 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb806c750 con 0x7f5eb80ae5a0 2023-11-24T06:29:39.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.229+0000 7f5eb7fff640 1 --2- 172.21.15.64:0/4265305963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb815e430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80cfea0 tx=0x7f5ea802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca81699ac276381 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5eb77fe640 1 -- 172.21.15.64:0/4265305963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8031870 con 0x7f5eb80ae5a0 2023-11-24T06:29:39.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5eb77fe640 1 -- 172.21.15.64:0/4265305963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea80319d0 con 0x7f5eb80ae5a0 2023-11-24T06:29:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5eb77fe640 1 -- 172.21.15.64:0/4265305963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8031d60 con 0x7f5eb80ae5a0 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/4265305963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80ae5a0 msgr2=0x7f5eb815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/4265305963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb815e430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80cfea0 tx=0x7f5ea802b910 comp rx=0 tx=0).stop 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/4265305963 shutdown_connections 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/4265305963 >> v2:172.21.15.162:3300/0 conn(0x7f5eb80b3ae0 0x7f5eb806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/4265305963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb815e430 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/4265305963 >> 172.21.15.64:0/4265305963 conn(0x7f5eb806a800 msgr2=0x7f5eb806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.230+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/4265305963 shutdown_connections 2023-11-24T06:29:39.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.231+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/4265305963 wait complete. 2023-11-24T06:29:39.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.232+0000 7f5ebe6c9640 1 Processor -- start 2023-11-24T06:29:39.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.232+0000 7f5ebe6c9640 1 -- start start 2023-11-24T06:29:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.233+0000 7f5ebe6c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eb806d120 0x7f5eb80cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.233+0000 7f5ebe6c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.233+0000 7f5ebe6c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80b3ae0 0x7f5eb80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.233+0000 7f5ebe6c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb81f70b0 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.233+0000 7f5ebe6c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80cb940 con 0x7f5eb80ae5a0 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.233+0000 7f5ebe6c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80cbc20 con 0x7f5eb806d120 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80b3ae0 0x7f5eb80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80b3ae0 0x7f5eb80ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49822/0 (socket says 172.21.15.64:49822) 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 -- 172.21.15.64:0/3029219364 learned_addr learned my addr 172.21.15.64:0/3029219364 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5eb7fff640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eb806d120 0x7f5eb80cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5eaffff640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 -- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eb806d120 msgr2=0x7f5eb80cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eb806d120 0x7f5eb80cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 -- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb80ae5a0 msgr2=0x7f5eb814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb814ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 -- 172.21.15.64:0/3029219364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea80094d0 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5eb7fff640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eb806d120 0x7f5eb80cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:39.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5eaffff640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:39.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.234+0000 7f5ebcc66640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80b3ae0 0x7f5eb80ce120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000c780 tx=0x7f5eb000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.235+0000 7f5eb57fa640 1 -- 172.21.15.64:0/3029219364 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003d80 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.235+0000 7f5eb57fa640 1 -- 172.21.15.64:0/3029219364 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb000ce90 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.235+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb80cbf00 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.235+0000 7f5eb57fa640 1 -- 172.21.15.64:0/3029219364 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00052b0 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.235+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb80ccc40 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.236+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb80cc420 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.237+0000 7f5eb57fa640 1 -- 172.21.15.64:0/3029219364 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5eb0005410 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.238+0000 7f5eb57fa640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5e9806ae40 0x7f5e9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.238+0000 7f5eb57fa640 1 -- 172.21.15.64:0/3029219364 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5eb0088bd0 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.238+0000 7f5eb7fff640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5e9806ae40 0x7f5e9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.239+0000 7f5eb7fff640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5e9806ae40 0x7f5e9806d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8036000 tx=0x7f5ea803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.242+0000 7f5eb57fa640 1 -- 172.21.15.64:0/3029219364 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5eb0053ee0 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.435+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5eb80675d0 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.441+0000 7f5eb57fa640 1 -- 172.21.15.64:0/3029219364 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5eb0017020 con 0x7f5eb80b3ae0 2023-11-24T06:29:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.444+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5e9806ae40 msgr2=0x7f5e9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.444+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5e9806ae40 0x7f5e9806d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8036000 tx=0x7f5ea803c040 comp rx=0 tx=0).stop 2023-11-24T06:29:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.444+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80b3ae0 msgr2=0x7f5eb80ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.444+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80b3ae0 0x7f5eb80ce120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000c780 tx=0x7f5eb000cc40 comp rx=0 tx=0).stop 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 shutdown_connections 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f5e9806ae40 0x7f5e9806d300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb80b3ae0 0x7f5eb80ce120 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb80ae5a0 0x7f5eb814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 --2- 172.21.15.64:0/3029219364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eb806d120 0x7f5eb80cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 >> 172.21.15.64:0/3029219364 conn(0x7f5eb806a800 msgr2=0x7f5eb80cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 shutdown_connections 2023-11-24T06:29:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.445+0000 7f5ebe6c9640 1 -- 172.21.15.64:0/3029219364 wait complete. 2023-11-24T06:29:39.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2023-11-24T06:29:39.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.680+0000 7f0399221640 1 Processor -- start 2023-11-24T06:29:39.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.680+0000 7f0399221640 1 -- start start 2023-11-24T06:29:39.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.682+0000 7f0399221640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0394143c40 0x7f0394071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.682+0000 7f0399221640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940ccc10 0x7f039406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.682+0000 7f0399221640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0394070e50 con 0x7f03940717d0 2023-11-24T06:29:39.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.682+0000 7f0399221640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03940723c0 con 0x7f0394143c40 2023-11-24T06:29:39.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.682+0000 7f0399221640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0394144020 con 0x7f03940ccc10 2023-11-24T06:29:39.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.683+0000 7f0398a20640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940ccc10 0x7f039406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.683+0000 7f0393fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0394143c40 0x7f0394071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.683+0000 7f0393fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0394143c40 0x7f0394071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:48394/0 (socket says 172.21.15.64:48394) 2023-11-24T06:29:39.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.683+0000 7f0393fff640 1 -- 172.21.15.64:0/2804430756 learned_addr learned my addr 172.21.15.64:0/2804430756 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:39.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.684+0000 7f0393fff640 1 -- 172.21.15.64:0/2804430756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940ccc10 msgr2=0x7f039406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:39.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.684+0000 7f0393fff640 1 --2- 172.21.15.64:0/2804430756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940ccc10 0x7f039406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.684+0000 7f0393fff640 1 -- 172.21.15.64:0/2804430756 >> v1:172.21.15.162:6789/0 conn(0x7f03940717d0 legacy=0x7f03940cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:39.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.684+0000 7f0393fff640 1 -- 172.21.15.64:0/2804430756 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394071e00 con 0x7f0394143c40 2023-11-24T06:29:39.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.685+0000 7f0393fff640 1 --2- 172.21.15.64:0/2804430756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0394143c40 0x7f0394071290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0384009e50 tx=0x7f038402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643cd3b399ebbb73 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0392ffd640 1 -- 172.21.15.64:0/2804430756 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038402ca70 con 0x7f0394143c40 2023-11-24T06:29:39.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0392ffd640 1 -- 172.21.15.64:0/2804430756 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038402cbd0 con 0x7f0394143c40 2023-11-24T06:29:39.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0392ffd640 1 -- 172.21.15.64:0/2804430756 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03840343f0 con 0x7f0394143c40 2023-11-24T06:29:39.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0399221640 1 -- 172.21.15.64:0/2804430756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0394143c40 msgr2=0x7f0394071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0399221640 1 --2- 172.21.15.64:0/2804430756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0394143c40 0x7f0394071290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0384009e50 tx=0x7f038402bed0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0399221640 1 -- 172.21.15.64:0/2804430756 shutdown_connections 2023-11-24T06:29:39.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0399221640 1 --2- 172.21.15.64:0/2804430756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940ccc10 0x7f039406d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0399221640 1 --2- 172.21.15.64:0/2804430756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0394143c40 0x7f0394071290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.686+0000 7f0399221640 1 -- 172.21.15.64:0/2804430756 >> 172.21.15.64:0/2804430756 conn(0x7f039406a800 msgr2=0x7f039406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.687+0000 7f0399221640 1 -- 172.21.15.64:0/2804430756 shutdown_connections 2023-11-24T06:29:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.687+0000 7f0399221640 1 -- 172.21.15.64:0/2804430756 wait complete. 2023-11-24T06:29:39.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.688+0000 7f0399221640 1 Processor -- start 2023-11-24T06:29:39.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.688+0000 7f0399221640 1 -- start start 2023-11-24T06:29:39.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.689+0000 7f0399221640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940717d0 0x7f03941e0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.689+0000 7f0399221640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940ccc10 0x7f039406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.689+0000 7f0399221640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0394143c40 0x7f03941e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.689+0000 7f0399221640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03941e19d0 con 0x7f0394143c40 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.689+0000 7f0399221640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03941e0dc0 con 0x7f03940ccc10 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.689+0000 7f0399221640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03941e10a0 con 0x7f03940717d0 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0394143c40 0x7f03941e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0394143c40 0x7f03941e3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49832/0 (socket says 172.21.15.64:49832) 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 -- 172.21.15.64:0/966002516 learned_addr learned my addr 172.21.15.64:0/966002516 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f03937fe640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940ccc10 0x7f039406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0393fff640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940717d0 0x7f03941e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 -- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940717d0 msgr2=0x7f03941e0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940717d0 0x7f03941e0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 -- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940ccc10 msgr2=0x7f039406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940ccc10 0x7f039406d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.690+0000 7f0398a20640 1 -- 172.21.15.64:0/966002516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384009b30 con 0x7f0394143c40 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.691+0000 7f0398a20640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0394143c40 0x7f03941e3cd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e7f0 tx=0x7f038c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.691+0000 7f0390ff9640 1 -- 172.21.15.64:0/966002516 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c009510 con 0x7f0394143c40 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.691+0000 7f0390ff9640 1 -- 172.21.15.64:0/966002516 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c00ee40 con 0x7f0394143c40 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.691+0000 7f0390ff9640 1 -- 172.21.15.64:0/966002516 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c00cb80 con 0x7f0394143c40 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.691+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03941e1380 con 0x7f0394143c40 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.691+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03941e0ab0 con 0x7f0394143c40 2023-11-24T06:29:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.692+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03941e1b40 con 0x7f0394143c40 2023-11-24T06:29:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.693+0000 7f0390ff9640 1 -- 172.21.15.64:0/966002516 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f038c00cce0 con 0x7f0394143c40 2023-11-24T06:29:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.694+0000 7f0390ff9640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f038006ae70 0x7f038006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.694+0000 7f0390ff9640 1 -- 172.21.15.64:0/966002516 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f038c0893c0 con 0x7f0394143c40 2023-11-24T06:29:39.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.694+0000 7f0393fff640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f038006ae70 0x7f038006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.695+0000 7f0393fff640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f038006ae70 0x7f038006d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0384035000 tx=0x7f0384030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.698+0000 7f0390ff9640 1 -- 172.21.15.64:0/966002516 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f038c054770 con 0x7f0394143c40 2023-11-24T06:29:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.954+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7f0394067450 con 0x7f0394143c40 2023-11-24T06:29:39.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.982+0000 7f0390ff9640 1 -- 172.21.15.64:0/966002516 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v127) v1 ==== 167+0+0 (secure 0 0 0) 0x7f038c057d90 con 0x7f0394143c40 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.985+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f038006ae70 msgr2=0x7f038006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.985+0000 7f0399221640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f038006ae70 0x7f038006d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0384035000 tx=0x7f0384030000 comp rx=0 tx=0).stop 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.985+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0394143c40 msgr2=0x7f03941e3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.985+0000 7f0399221640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0394143c40 0x7f03941e3cd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e7f0 tx=0x7f038c00ebf0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 shutdown_connections 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f038006ae70 0x7f038006d330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0394143c40 0x7f03941e3cd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940ccc10 0x7f039406d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 --2- 172.21.15.64:0/966002516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03940717d0 0x7f03941e0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 >> 172.21.15.64:0/966002516 conn(0x7f039406a800 msgr2=0x7f039406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:39.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 shutdown_connections 2023-11-24T06:29:39.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:39.986+0000 7f0399221640 1 -- 172.21.15.64:0/966002516 wait complete. 2023-11-24T06:29:40.009 INFO:tasks.cephfs_test_runner:test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) ... ERROR 2023-11-24T06:29:40.009 INFO:tasks.cephfs_test_runner: 2023-11-24T06:29:40.009 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:29:40.009 INFO:tasks.cephfs_test_runner:ERROR: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 8021, in test_volumes_module_finisher_thread 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner: subvol1, subvol2, subvol3 = self._generate_random_subvolume_name(3) 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:AttributeError: 'TestPerModuleFinsherThread' object has no attribute '_generate_random_subvolume_name' 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner: 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:Ran 1 test in 75.041s 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner: 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner: 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:29:40.010 INFO:tasks.cephfs_test_runner:ERROR: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T06:29:40.011 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:29:40.011 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:29:40.011 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 8021, in test_volumes_module_finisher_thread 2023-11-24T06:29:40.011 INFO:tasks.cephfs_test_runner: subvol1, subvol2, subvol3 = self._generate_random_subvolume_name(3) 2023-11-24T06:29:40.011 INFO:tasks.cephfs_test_runner:AttributeError: 'TestPerModuleFinsherThread' object has no attribute '_generate_random_subvolume_name' 2023-11-24T06:29:40.011 INFO:tasks.cephfs_test_runner: 2023-11-24T06:29:40.011 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T06:29:40.087 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=faf45a1ab6264ac29c1d0f6e1a9cfd6d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T06:29:40.088 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T06:29:40.098 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-24T06:29:40.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:40.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.338+0000 7f4b974ef640 1 Processor -- start 2023-11-24T06:29:40.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.338+0000 7f4b974ef640 1 -- start start 2023-11-24T06:29:40.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.339+0000 7f4b974ef640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b90160920 0x7f4b900b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.339+0000 7f4b974ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b900cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b974ef640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b900d0a10 con 0x7f4b90161010 2023-11-24T06:29:40.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b974ef640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b900d03b0 con 0x7f4b90160920 2023-11-24T06:29:40.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b974ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900d1f50 con 0x7f4b900cf9f0 2023-11-24T06:29:40.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b964ed640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b90160920 0x7f4b900b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b964ed640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b90160920 0x7f4b900b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49844/0 (socket says 172.21.15.64:49844) 2023-11-24T06:29:40.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b964ed640 1 -- 172.21.15.64:0/2075737405 learned_addr learned my addr 172.21.15.64:0/2075737405 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:40.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b95cec640 1 --2- 172.21.15.64:0/2075737405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b900cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b964ed640 1 -- 172.21.15.64:0/2075737405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 msgr2=0x7f4b900cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:40.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b964ed640 1 --2- 172.21.15.64:0/2075737405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b900cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b964ed640 1 -- 172.21.15.64:0/2075737405 >> v1:172.21.15.162:6789/0 conn(0x7f4b90161010 legacy=0x7f4b901614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:40.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.340+0000 7f4b964ed640 1 -- 172.21.15.64:0/2075737405 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900cf080 con 0x7f4b90160920 2023-11-24T06:29:40.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.342+0000 7f4b964ed640 1 --2- 172.21.15.64:0/2075737405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b3f30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009870 tx=0x7f4b8c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f22edf5a0324b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:40.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.342+0000 7f4b954eb640 1 -- 172.21.15.64:0/2075737405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c031640 con 0x7f4b90160920 2023-11-24T06:29:40.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.342+0000 7f4b954eb640 1 -- 172.21.15.64:0/2075737405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c0317a0 con 0x7f4b90160920 2023-11-24T06:29:40.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.342+0000 7f4b954eb640 1 -- 172.21.15.64:0/2075737405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c031b30 con 0x7f4b90160920 2023-11-24T06:29:40.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 -- 172.21.15.64:0/2075737405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 msgr2=0x7f4b900b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 --2- 172.21.15.64:0/2075737405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b3f30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009870 tx=0x7f4b8c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 -- 172.21.15.64:0/2075737405 shutdown_connections 2023-11-24T06:29:40.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 --2- 172.21.15.64:0/2075737405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b900cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 --2- 172.21.15.64:0/2075737405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b3f30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 -- 172.21.15.64:0/2075737405 >> 172.21.15.64:0/2075737405 conn(0x7f4b900d3090 msgr2=0x7f4b900d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:40.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 -- 172.21.15.64:0/2075737405 shutdown_connections 2023-11-24T06:29:40.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.343+0000 7f4b974ef640 1 -- 172.21.15.64:0/2075737405 wait complete. 2023-11-24T06:29:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.346+0000 7f4b974ef640 1 Processor -- start 2023-11-24T06:29:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.346+0000 7f4b974ef640 1 -- start start 2023-11-24T06:29:40.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b974ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b901543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b974ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b974ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b90161010 0x7f4b900b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b974ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900cbb20 con 0x7f4b90160920 2023-11-24T06:29:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b974ef640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900caf10 con 0x7f4b90161010 2023-11-24T06:29:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b974ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900cb1f0 con 0x7f4b900cf9f0 2023-11-24T06:29:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b95cec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b95cec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49848/0 (socket says 172.21.15.64:49848) 2023-11-24T06:29:40.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.347+0000 7f4b95cec640 1 -- 172.21.15.64:0/399639269 learned_addr learned my addr 172.21.15.64:0/399639269 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:40.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b96cee640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b90161010 0x7f4b900b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b964ed640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b901543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b95cec640 1 -- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 msgr2=0x7f4b901543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b95cec640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b901543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b95cec640 1 -- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b90161010 msgr2=0x7f4b900b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b95cec640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b90161010 0x7f4b900b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b95cec640 1 -- 172.21.15.64:0/399639269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0094d0 con 0x7f4b90160920 2023-11-24T06:29:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b96cee640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b90161010 0x7f4b900b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b964ed640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b901543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.348+0000 7f4b95cec640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b7090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c7f0 tx=0x7f4b8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.349+0000 7f4b86ffd640 1 -- 172.21.15.64:0/399639269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80004220 con 0x7f4b90160920 2023-11-24T06:29:40.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.349+0000 7f4b86ffd640 1 -- 172.21.15.64:0/399639269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b800043a0 con 0x7f4b90160920 2023-11-24T06:29:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.349+0000 7f4b86ffd640 1 -- 172.21.15.64:0/399639269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80007860 con 0x7f4b90160920 2023-11-24T06:29:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.349+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b900cb4d0 con 0x7f4b90160920 2023-11-24T06:29:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.349+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b900cab70 con 0x7f4b90160920 2023-11-24T06:29:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.350+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b90067450 con 0x7f4b90160920 2023-11-24T06:29:40.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.351+0000 7f4b86ffd640 1 -- 172.21.15.64:0/399639269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b80007bd0 con 0x7f4b90160920 2023-11-24T06:29:40.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.352+0000 7f4b86ffd640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b7406ae40 0x7f4b7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.352+0000 7f4b86ffd640 1 -- 172.21.15.64:0/399639269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4b80089380 con 0x7f4b90160920 2023-11-24T06:29:40.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.353+0000 7f4b964ed640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b7406ae40 0x7f4b7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.354+0000 7f4b964ed640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b7406ae40 0x7f4b7406d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c036000 tx=0x7f4b8c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:40.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.357+0000 7f4b86ffd640 1 -- 172.21.15.64:0/399639269 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b80054690 con 0x7f4b90160920 2023-11-24T06:29:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.553+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b90161e90 con 0x7f4b90160920 2023-11-24T06:29:40.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.556+0000 7f4b86ffd640 1 -- 172.21.15.64:0/399639269 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12962 (secure 0 0 0) 0x7f4b80057cb0 con 0x7f4b90160920 2023-11-24T06:29:40.559 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:40.559 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:38.962984+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6836/2541249514":"2023-11-25T06:29:35.901267+0000","172.21.15.64:6834/2541249514":"2023-11-25T06:29:35.901267+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b7406ae40 msgr2=0x7f4b7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b7406ae40 0x7f4b7406d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c036000 tx=0x7f4b8c03c040 comp rx=0 tx=0).stop 2023-11-24T06:29:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 msgr2=0x7f4b900b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b7090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c7f0 tx=0x7f4b8000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 shutdown_connections 2023-11-24T06:29:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4b7406ae40 0x7f4b7406d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b90161010 0x7f4b900b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b90160920 0x7f4b900b7090 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 --2- 172.21.15.64:0/399639269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b900cf9f0 0x7f4b901543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.558+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 >> 172.21.15.64:0/399639269 conn(0x7f4b900d3090 msgr2=0x7f4b9006c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.559+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 shutdown_connections 2023-11-24T06:29:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.559+0000 7f4b974ef640 1 -- 172.21.15.64:0/399639269 wait complete. 2023-11-24T06:29:40.579 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:27:35.179412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:40.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:40.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.797+0000 7f7c65fcb640 1 Processor -- start 2023-11-24T06:29:40.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.798+0000 7f7c65fcb640 1 -- start start 2023-11-24T06:29:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.799+0000 7f7c65fcb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c600cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.799+0000 7f7c65fcb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c600b31d0 0x7f7c600b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.799+0000 7f7c65fcb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c600d0320 con 0x7f7c6015d4b0 2023-11-24T06:29:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.799+0000 7f7c65fcb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c600d1c60 con 0x7f7c600b31d0 2023-11-24T06:29:40.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.799+0000 7f7c65fcb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600b2730 con 0x7f7c600cf9a0 2023-11-24T06:29:40.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.800+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c600cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.800+0000 7f7c657ca640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c600b31d0 0x7f7c600b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.800+0000 7f7c657ca640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c600b31d0 0x7f7c600b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:48416/0 (socket says 172.21.15.64:48416) 2023-11-24T06:29:40.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.800+0000 7f7c657ca640 1 -- 172.21.15.64:0/316774613 learned_addr learned my addr 172.21.15.64:0/316774613 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:40.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.801+0000 7f7c657ca640 1 -- 172.21.15.64:0/316774613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 msgr2=0x7f7c600cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:40.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.801+0000 7f7c657ca640 1 --2- 172.21.15.64:0/316774613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c600cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.801+0000 7f7c657ca640 1 -- 172.21.15.64:0/316774613 >> v1:172.21.15.162:6789/0 conn(0x7f7c6015d4b0 legacy=0x7f7c600d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.801+0000 7f7c657ca640 1 -- 172.21.15.64:0/316774613 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600d1ec0 con 0x7f7c600b31d0 2023-11-24T06:29:40.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.803+0000 7f7c657ca640 1 --2- 172.21.15.64:0/316774613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c600b31d0 0x7f7c600b3660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e80 tx=0x7f7c5802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=953a0f4fd8a0c1bd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:40.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.803+0000 7f7c5f7fe640 1 -- 172.21.15.64:0/316774613 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5802cae0 con 0x7f7c600b31d0 2023-11-24T06:29:40.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.803+0000 7f7c5f7fe640 1 -- 172.21.15.64:0/316774613 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5802cc40 con 0x7f7c600b31d0 2023-11-24T06:29:40.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c5f7fe640 1 -- 172.21.15.64:0/316774613 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c580343f0 con 0x7f7c600b31d0 2023-11-24T06:29:40.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 -- 172.21.15.64:0/316774613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c600b31d0 msgr2=0x7f7c600b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/316774613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c600b31d0 0x7f7c600b3660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e80 tx=0x7f7c5802bf40 comp rx=0 tx=0).stop 2023-11-24T06:29:40.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 -- 172.21.15.64:0/316774613 shutdown_connections 2023-11-24T06:29:40.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/316774613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c600cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/316774613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c600b31d0 0x7f7c600b3660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 -- 172.21.15.64:0/316774613 >> 172.21.15.64:0/316774613 conn(0x7f7c600d3000 msgr2=0x7f7c600d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 -- 172.21.15.64:0/316774613 shutdown_connections 2023-11-24T06:29:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.804+0000 7f7c65fcb640 1 -- 172.21.15.64:0/316774613 wait complete. 2023-11-24T06:29:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.806+0000 7f7c65fcb640 1 Processor -- start 2023-11-24T06:29:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.806+0000 7f7c65fcb640 1 -- start start 2023-11-24T06:29:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.807+0000 7f7c65fcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c600b31d0 0x7f7c6014feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.807+0000 7f7c65fcb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c6015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.807+0000 7f7c65fcb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c6015d4b0 0x7f7c600b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.807+0000 7f7c65fcb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600b5cc0 con 0x7f7c600b31d0 2023-11-24T06:29:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.807+0000 7f7c65fcb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600b5fa0 con 0x7f7c6015d4b0 2023-11-24T06:29:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.807+0000 7f7c65fcb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600b6280 con 0x7f7c600cf9a0 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c600b31d0 0x7f7c6014feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c600b31d0 0x7f7c6014feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49862/0 (socket says 172.21.15.64:49862) 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 -- 172.21.15.64:0/3787015784 learned_addr learned my addr 172.21.15.64:0/3787015784 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c5ffff640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c6015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c657ca640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c6015d4b0 0x7f7c600b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 -- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 msgr2=0x7f7c6015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c6015fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 -- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c6015d4b0 msgr2=0x7f7c600b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c6015d4b0 0x7f7c600b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.808+0000 7f7c64fc9640 1 -- 172.21.15.64:0/3787015784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c58009b30 con 0x7f7c600b31d0 2023-11-24T06:29:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.809+0000 7f7c64fc9640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c600b31d0 0x7f7c6014feb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c400 tx=0x7f7c5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.809+0000 7f7c5d7fa640 1 -- 172.21.15.64:0/3787015784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540052d0 con 0x7f7c600b31d0 2023-11-24T06:29:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.809+0000 7f7c5d7fa640 1 -- 172.21.15.64:0/3787015784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c54005430 con 0x7f7c600b31d0 2023-11-24T06:29:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.809+0000 7f7c5d7fa640 1 -- 172.21.15.64:0/3787015784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54003830 con 0x7f7c600b31d0 2023-11-24T06:29:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.809+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c600b6560 con 0x7f7c600b31d0 2023-11-24T06:29:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.809+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c601503f0 con 0x7f7c600b31d0 2023-11-24T06:29:40.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.810+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c60150580 con 0x7f7c600b31d0 2023-11-24T06:29:40.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.811+0000 7f7c5d7fa640 1 -- 172.21.15.64:0/3787015784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c54003990 con 0x7f7c600b31d0 2023-11-24T06:29:40.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.813+0000 7f7c5d7fa640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:40.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.813+0000 7f7c5d7fa640 1 -- 172.21.15.64:0/3787015784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7c54088890 con 0x7f7c600b31d0 2023-11-24T06:29:40.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.813+0000 7f7c5ffff640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:40.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.814+0000 7f7c5ffff640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c4406ae40 0x7f7c4406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c60160fd0 tx=0x7f7c4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:40.818+0000 7f7c5d7fa640 1 -- 172.21.15.64:0/3787015784 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c60150580 con 0x7f7c600b31d0 2023-11-24T06:29:41.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.013+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c60150580 con 0x7f7c600b31d0 2023-11-24T06:29:41.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.014+0000 7f7c5d7fa640 1 -- 172.21.15.64:0/3787015784 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c60150580 con 0x7f7c600b31d0 2023-11-24T06:29:41.017 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T06:29:41.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c4406ae40 msgr2=0x7f7c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c4406ae40 0x7f7c4406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c60160fd0 tx=0x7f7c4c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c600b31d0 msgr2=0x7f7c6014feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c600b31d0 0x7f7c6014feb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c400 tx=0x7f7c5400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 shutdown_connections 2023-11-24T06:29:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c6015d4b0 0x7f7c600b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c600cf9a0 0x7f7c6015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.017+0000 7f7c65fcb640 1 --2- 172.21.15.64:0/3787015784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c600b31d0 0x7f7c6014feb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.018+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 >> 172.21.15.64:0/3787015784 conn(0x7f7c600d3000 msgr2=0x7f7c600b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.018+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 shutdown_connections 2023-11-24T06:29:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.018+0000 7f7c65fcb640 1 -- 172.21.15.64:0/3787015784 wait complete. 2023-11-24T06:29:41.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-24T06:29:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.262+0000 7f8940973640 1 Processor -- start 2023-11-24T06:29:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.262+0000 7f8940973640 1 -- start start 2023-11-24T06:29:41.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f8940973640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 0x7f893c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f8940973640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c0d0a10 con 0x7f893c0cf9f0 2023-11-24T06:29:41.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f8940973640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c161e30 con 0x7f893c0cfe70 2023-11-24T06:29:41.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f8940973640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0d1f50 con 0x7f893c160920 2023-11-24T06:29:41.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f8933fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f893c0cf9f0 0x7f893c161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43744/0 (socket says 172.21.15.64:43744) 2023-11-24T06:29:41.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f8933fff640 1 -- 172.21.15.64:0/362765080 learned_addr learned my addr 172.21.15.64:0/362765080 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888494114 0 0) 0x7f893c0d0a10 con 0x7f893c0cf9f0 2023-11-24T06:29:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f893b7fe640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 0x7f893c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.264+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8928003520 con 0x7f893c0cf9f0 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3802863244 0 0) 0x7f8928003520 con 0x7f893c0cf9f0 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 msgr2=0x7f893c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 0x7f893c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 >> v1:172.21.15.162:6789/0 conn(0x7f893c0cfe70 legacy=0x7f893c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0cf080 con 0x7f893c0cf9f0 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f8924001670 con 0x7f893c0cf9f0 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 >> v1:172.21.15.64:6789/0 conn(0x7f893c0cf9f0 legacy=0x7f893c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f8928003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 0x7f89280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8928004d00 0x7f8928005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0d1f50 con 0x7f89280039a0 2023-11-24T06:29:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892800ba30 con 0x7f8928004d00 2023-11-24T06:29:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892800bbf0 con 0x7f893c160920 2023-11-24T06:29:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.265+0000 7f893b7fe640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f8928003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f8933fff640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 0x7f89280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f893bfff640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8928004d00 0x7f8928005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f893b7fe640 1 -- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 msgr2=0x7f89280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f893b7fe640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c160920 0x7f89280047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f893b7fe640 1 -- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8928004d00 msgr2=0x7f8928005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f893b7fe640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8928004d00 0x7f8928005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f893b7fe640 1 -- 172.21.15.64:0/362765080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928003520 con 0x7f89280039a0 2023-11-24T06:29:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.266+0000 7f893bfff640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8928004d00 0x7f8928005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.267+0000 7f893b7fe640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f8928003dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f892c009a90 tx=0x7f89280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e21e0bc6bff05e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.267+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c02c680 con 0x7f89280039a0 2023-11-24T06:29:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.267+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c02c7e0 con 0x7f89280039a0 2023-11-24T06:29:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.267+0000 7f893affd640 1 -- 172.21.15.64:0/362765080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c02cb70 con 0x7f89280039a0 2023-11-24T06:29:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.268+0000 7f8940973640 1 -- 172.21.15.64:0/362765080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 msgr2=0x7f8928003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.268+0000 7f8940973640 1 --2- 172.21.15.64:0/362765080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f8928003dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f892c009a90 tx=0x7f89280057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.268+0000 7f893bfff640 1 -- 172.21.15.64:0/362765080 reap_dead start 2023-11-24T06:29:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.268+0000 7f8940973640 1 -- 172.21.15.64:0/362765080 shutdown_connections 2023-11-24T06:29:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.268+0000 7f8940973640 1 -- 172.21.15.64:0/362765080 >> 172.21.15.64:0/362765080 conn(0x7f893c0d3090 msgr2=0x7f893c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.268+0000 7f8940973640 1 -- 172.21.15.64:0/362765080 shutdown_connections 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.268+0000 7f8940973640 1 -- 172.21.15.64:0/362765080 wait complete. 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.270+0000 7f8940973640 1 Processor -- start 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.270+0000 7f8940973640 1 -- start start 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f8940973640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f893c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f8940973640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c153140 0x7f893c154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f8940973640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f893c0cada0 0x7f893c0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f8940973640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1535c0 con 0x7f89280039a0 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f8940973640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c155000 con 0x7f893c0cada0 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f8940973640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1537d0 con 0x7f893c153140 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f893c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f893c0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49870/0 (socket says 172.21.15.64:49870) 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 -- 172.21.15.64:0/4005636290 learned_addr learned my addr 172.21.15.64:0/4005636290 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 -- 172.21.15.64:0/4005636290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c153140 msgr2=0x7f893c154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c153140 0x7f893c154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 -- 172.21.15.64:0/4005636290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f893c0cada0 msgr2=0x7f893c0b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f893c0cada0 0x7f893c0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.271+0000 7f893b7fe640 1 -- 172.21.15.64:0/4005636290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c153ab0 con 0x7f89280039a0 2023-11-24T06:29:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.272+0000 7f893b7fe640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f893c0ca160 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f892c037040 tx=0x7f892c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.272+0000 7f8938ff9640 1 -- 172.21.15.64:0/4005636290 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c003810 con 0x7f89280039a0 2023-11-24T06:29:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.272+0000 7f8938ff9640 1 -- 172.21.15.64:0/4005636290 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c003970 con 0x7f89280039a0 2023-11-24T06:29:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.272+0000 7f8938ff9640 1 -- 172.21.15.64:0/4005636290 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c003d20 con 0x7f89280039a0 2023-11-24T06:29:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.272+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c153d90 con 0x7f89280039a0 2023-11-24T06:29:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.272+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c0b69b0 con 0x7f89280039a0 2023-11-24T06:29:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.274+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c153580 con 0x7f89280039a0 2023-11-24T06:29:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.274+0000 7f8938ff9640 1 -- 172.21.15.64:0/4005636290 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f892c034440 con 0x7f89280039a0 2023-11-24T06:29:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.275+0000 7f8938ff9640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f891c06ae40 0x7f891c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.275+0000 7f8938ff9640 1 -- 172.21.15.64:0/4005636290 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f892c0ab190 con 0x7f89280039a0 2023-11-24T06:29:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.276+0000 7f8933fff640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f891c06ae40 0x7f891c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.276+0000 7f8933fff640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f891c06ae40 0x7f891c06d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8924000f60 tx=0x7f8924009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.280+0000 7f8938ff9640 1 -- 172.21.15.64:0/4005636290 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f892c076710 con 0x7f89280039a0 2023-11-24T06:29:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.526+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f893c067450 con 0x7f89280039a0 2023-11-24T06:29:41.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.546+0000 7f8938ff9640 1 -- 172.21.15.64:0/4005636290 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v5) v1 ==== 81+0+466145 (secure 0 0 0) 0x7f892c079d30 con 0x7f89280039a0 2023-11-24T06:29:41.552 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:41.552 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:41.552 INFO:teuthology.orchestra.run.smithi064.stdout: "epoch": 5, 2023-11-24T06:29:41.553 INFO:teuthology.orchestra.run.smithi064.stdout: "active_gid": 4107, 2023-11-24T06:29:41.553 INFO:teuthology.orchestra.run.smithi064.stdout: "active_name": "y", 2023-11-24T06:29:41.553 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addrs": { 2023-11-24T06:29:41.553 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-11-24T06:29:41.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.553 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-11-24T06:29:41.553 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6832", 2023-11-24T06:29:41.554 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1000290349 2023-11-24T06:29:41.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T06:29:41.554 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6833", 2023-11-24T06:29:41.554 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1000290349 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addr": "172.21.15.64:6833/1000290349", 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: "active_change": "2023-11-24T06:27:27.073346+0000", 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: "active_mgr_features": 4540138322906710015, 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: "available": true, 2023-11-24T06:29:41.555 INFO:teuthology.orchestra.run.smithi064.stdout: "standbys": [ 2023-11-24T06:29:41.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.556 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 4108, 2023-11-24T06:29:41.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "x", 2023-11-24T06:29:41.556 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540138322906710015, 2023-11-24T06:29:41.556 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2023-11-24T06:29:41.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2023-11-24T06:29:41.557 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.557 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.557 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.557 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:41.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:41.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:41.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.558 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2023-11-24T06:29:41.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.559 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.561 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.561 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.561 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.563 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.563 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.564 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.566 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.566 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.568 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.570 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.570 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.570 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.570 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.570 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.571 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.571 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.571 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.572 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.572 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.573 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.574 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2023-11-24T06:29:41.574 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2023-11-24T06:29:41.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.574 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2023-11-24T06:29:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.577 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2023-11-24T06:29:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2023-11-24T06:29:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2023-11-24T06:29:41.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2023-11-24T06:29:41.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.580 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.580 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.580 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.580 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2023-11-24T06:29:41.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.582 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2023-11-24T06:29:41.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2023-11-24T06:29:41.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.585 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2023-11-24T06:29:41.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.586 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2023-11-24T06:29:41.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2023-11-24T06:29:41.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2023-11-24T06:29:41.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2023-11-24T06:29:41.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.592 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.594 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2023-11-24T06:29:41.594 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2023-11-24T06:29:41.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.596 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2023-11-24T06:29:41.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2023-11-24T06:29:41.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:41.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2023-11-24T06:29:41.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2023-11-24T06:29:41.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2023-11-24T06:29:41.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2023-11-24T06:29:41.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:41.602 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.602 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2023-11-24T06:29:41.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2023-11-24T06:29:41.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2023-11-24T06:29:41.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T06:29:41.604 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.605 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:41.606 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-11-24T06:29:41.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-11-24T06:29:41.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-24T06:29:41.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T06:29:41.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.608 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.612 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.613 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.613 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.613 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.613 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.613 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:41.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.617 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-11-24T06:29:41.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-11-24T06:29:41.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2023-11-24T06:29:41.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2023-11-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2023-11-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2023-11-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2023-11-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2023-11-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2023-11-24T06:29:41.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.638 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2023-11-24T06:29:41.638 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.638 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.638 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-24T06:29:41.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-24T06:29:41.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.646 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2023-11-24T06:29:41.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2023-11-24T06:29:41.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-24T06:29:41.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:41.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.650 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2023-11-24T06:29:41.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2023-11-24T06:29:41.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.652 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2023-11-24T06:29:41.652 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2023-11-24T06:29:41.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2023-11-24T06:29:41.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.656 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2023-11-24T06:29:41.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2023-11-24T06:29:41.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.656 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2023-11-24T06:29:41.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.659 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2023-11-24T06:29:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2023-11-24T06:29:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-24T06:29:41.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2023-11-24T06:29:41.664 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.665 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2023-11-24T06:29:41.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2023-11-24T06:29:41.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2023-11-24T06:29:41.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2023-11-24T06:29:41.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-24T06:29:41.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2023-11-24T06:29:41.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2023-11-24T06:29:41.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.677 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2023-11-24T06:29:41.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2023-11-24T06:29:41.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:41.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2023-11-24T06:29:41.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2023-11-24T06:29:41.682 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.682 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.684 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2023-11-24T06:29:41.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.684 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.684 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2023-11-24T06:29:41.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:41.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.688 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.690 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2023-11-24T06:29:41.690 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.690 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2023-11-24T06:29:41.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.696 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.697 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.698 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.701 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2023-11-24T06:29:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.710 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.712 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.714 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2023-11-24T06:29:41.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2023-11-24T06:29:41.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.714 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-24T06:29:41.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.715 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2023-11-24T06:29:41.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-24T06:29:41.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.722 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-24T06:29:41.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2023-11-24T06:29:41.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2023-11-24T06:29:41.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.727 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.727 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2023-11-24T06:29:41.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-24T06:29:41.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.733 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2023-11-24T06:29:41.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-24T06:29:41.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2023-11-24T06:29:41.737 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.739 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.741 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.742 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.743 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.746 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2023-11-24T06:29:41.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.748 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-24T06:29:41.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-24T06:29:41.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.755 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-24T06:29:41.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-24T06:29:41.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.758 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.758 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-24T06:29:41.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.762 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-24T06:29:41.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-24T06:29:41.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.762 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.763 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.766 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-24T06:29:41.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-24T06:29:41.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.779 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-11-24T06:29:41.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.782 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.783 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.783 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-24T06:29:41.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-24T06:29:41.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.787 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-24T06:29:41.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-24T06:29:41.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2023-11-24T06:29:41.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-24T06:29:41.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-24T06:29:41.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.804 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.807 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-24T06:29:41.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.810 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2023-11-24T06:29:41.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.816 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.819 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-24T06:29:41.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.822 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.825 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-24T06:29:41.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.830 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.831 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.833 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.834 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.836 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.839 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-11-24T06:29:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2023-11-24T06:29:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2023-11-24T06:29:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.860 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2023-11-24T06:29:41.862 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2023-11-24T06:29:41.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.865 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.866 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.866 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.866 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.868 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.869 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.870 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.870 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.870 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.870 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.870 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.871 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2023-11-24T06:29:41.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.879 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2023-11-24T06:29:41.884 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-24T06:29:41.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.888 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.889 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.891 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.895 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.896 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2023-11-24T06:29:41.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.901 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2023-11-24T06:29:41.906 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.907 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:41.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.911 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.912 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.913 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.914 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.915 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.916 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.917 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2023-11-24T06:29:41.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2023-11-24T06:29:41.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.924 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.925 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-24T06:29:41.926 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.926 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.926 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.926 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2023-11-24T06:29:41.926 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2023-11-24T06:29:41.926 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:41.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.929 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.930 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.934 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.936 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.937 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.939 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.940 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.941 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.942 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.943 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.945 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.946 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.946 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.946 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.946 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.949 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.950 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.952 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.958 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2023-11-24T06:29:41.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2023-11-24T06:29:41.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.966 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.966 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.966 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.966 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2023-11-24T06:29:41.966 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.967 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.969 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.970 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.973 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.975 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2023-11-24T06:29:41.978 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.981 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.984 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:41.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.989 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.990 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.994 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.994 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:41.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:41.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:41.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.007 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2023-11-24T06:29:42.011 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.013 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.014 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.018 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.019 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.022 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.023 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.026 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.028 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.031 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.032 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.032 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.032 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.033 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.036 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.037 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:42.040 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.041 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.044 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.045 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-11-24T06:29:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.049 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.053 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.054 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.055 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2023-11-24T06:29:42.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.058 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.060 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2023-11-24T06:29:42.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.063 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.064 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.065 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.067 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.069 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.071 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.072 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.076 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-11-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-11-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.088 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.089 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.093 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.097 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.098 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2023-11-24T06:29:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.100 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.100 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.100 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.101 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.101 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.101 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.104 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-11-24T06:29:42.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-11-24T06:29:42.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.104 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.104 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.105 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.106 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.107 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.108 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.111 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.116 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.119 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.124 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.125 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.125 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "drive_group_interval": { 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "drive_group_interval", 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300.0", 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.131 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.132 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.133 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.135 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.137 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.139 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.141 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.142 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.143 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.146 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.149 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.149 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.150 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2023-11-24T06:29:42.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.154 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2023-11-24T06:29:42.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2023-11-24T06:29:42.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.167 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2023-11-24T06:29:42.168 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.172 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.173 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2023-11-24T06:29:42.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2023-11-24T06:29:42.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.180 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.184 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.186 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.190 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.192 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.193 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.195 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.199 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.203 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.207 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.212 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.213 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.218 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.219 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-24T06:29:42.221 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.221 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.221 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.221 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2023-11-24T06:29:42.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2023-11-24T06:29:42.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-24T06:29:42.222 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2023-11-24T06:29:42.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.245 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.248 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2023-11-24T06:29:42.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.251 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2023-11-24T06:29:42.252 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.254 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.258 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.260 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.262 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.263 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.264 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.266 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.268 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.269 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.270 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.271 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2023-11-24T06:29:42.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-24T06:29:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.276 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2023-11-24T06:29:42.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.279 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:42.282 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.284 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.285 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.288 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.289 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.290 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.292 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.294 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2023-11-24T06:29:42.295 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.296 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:42.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: "modules": [ 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: "iostat", 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: "nfs", 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: "restful" 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.298 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.299 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.300 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.301 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.302 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.304 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.306 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2023-11-24T06:29:42.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2023-11-24T06:29:42.310 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2023-11-24T06:29:42.314 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.317 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.319 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.320 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2023-11-24T06:29:42.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2023-11-24T06:29:42.322 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2023-11-24T06:29:42.323 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.324 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.326 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2023-11-24T06:29:42.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-24T06:29:42.328 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.330 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2023-11-24T06:29:42.331 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.332 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2023-11-24T06:29:42.333 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2023-11-24T06:29:42.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2023-11-24T06:29:42.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.339 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.343 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2023-11-24T06:29:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-24T06:29:42.347 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2023-11-24T06:29:42.349 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2023-11-24T06:29:42.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-24T06:29:42.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:42.354 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2023-11-24T06:29:42.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2023-11-24T06:29:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.358 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.358 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.358 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2023-11-24T06:29:42.358 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2023-11-24T06:29:42.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.358 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.359 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2023-11-24T06:29:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2023-11-24T06:29:42.361 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.362 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-24T06:29:42.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.365 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.367 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2023-11-24T06:29:42.368 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2023-11-24T06:29:42.369 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.370 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.371 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2023-11-24T06:29:42.372 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-24T06:29:42.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.375 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-24T06:29:42.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.377 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.378 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2023-11-24T06:29:42.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2023-11-24T06:29:42.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.382 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-24T06:29:42.383 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.384 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2023-11-24T06:29:42.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.392 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.395 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2023-11-24T06:29:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.399 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2023-11-24T06:29:42.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-24T06:29:42.406 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.408 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2023-11-24T06:29:42.408 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-24T06:29:42.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2023-11-24T06:29:42.410 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.410 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.410 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2023-11-24T06:29:42.410 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.410 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2023-11-24T06:29:42.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.419 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.420 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.425 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.427 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2023-11-24T06:29:42.430 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2023-11-24T06:29:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-24T06:29:42.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2023-11-24T06:29:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.442 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.445 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2023-11-24T06:29:42.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2023-11-24T06:29:42.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.446 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.446 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.446 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.446 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-24T06:29:42.448 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.449 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2023-11-24T06:29:42.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2023-11-24T06:29:42.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2023-11-24T06:29:42.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-24T06:29:42.455 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.458 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2023-11-24T06:29:42.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.460 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.463 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.465 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.465 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.468 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.469 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.473 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-24T06:29:42.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.478 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.478 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-24T06:29:42.482 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.483 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.486 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-24T06:29:42.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-24T06:29:42.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-24T06:29:42.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2023-11-24T06:29:42.502 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.503 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.506 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-24T06:29:42.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.511 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2023-11-24T06:29:42.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-24T06:29:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.518 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-24T06:29:42.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.525 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.526 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2023-11-24T06:29:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.528 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.531 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.532 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.534 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.535 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-24T06:29:42.537 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.538 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.541 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-24T06:29:42.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.545 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-24T06:29:42.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.550 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2023-11-24T06:29:42.554 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-11-24T06:29:42.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.558 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.559 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.562 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.563 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.564 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.566 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-11-24T06:29:42.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.573 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2023-11-24T06:29:42.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-11-24T06:29:42.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2023-11-24T06:29:42.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2023-11-24T06:29:42.580 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.582 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.583 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.588 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2023-11-24T06:29:42.591 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2023-11-24T06:29:42.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2023-11-24T06:29:42.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-24T06:29:42.598 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.604 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.608 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.611 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2023-11-24T06:29:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.616 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.617 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.622 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.628 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.631 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.634 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-11-24T06:29:42.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T06:29:42.637 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.641 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.645 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.649 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.650 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.650 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.650 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.650 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.650 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.659 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.662 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.662 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-24T06:29:42.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.665 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.666 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.667 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.669 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.669 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.669 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.674 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.674 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.674 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.674 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2023-11-24T06:29:42.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2023-11-24T06:29:42.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.682 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2023-11-24T06:29:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2023-11-24T06:29:42.686 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.688 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.705 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.707 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.711 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.717 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.718 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.722 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.725 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.729 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.730 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.733 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.734 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.734 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.734 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.734 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.735 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.739 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.747 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2023-11-24T06:29:42.747 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.748 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.749 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.750 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.752 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.753 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.756 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.757 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2023-11-24T06:29:42.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-24T06:29:42.759 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.760 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.765 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-24T06:29:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-24T06:29:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-24T06:29:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2023-11-24T06:29:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2023-11-24T06:29:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2023-11-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-11-24T06:29:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "drive_group_interval": { 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "drive_group_interval", 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300.0", 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2023-11-24T06:29:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2023-11-24T06:29:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2023-11-24T06:29:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2023-11-24T06:29:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f891c06ae40 msgr2=0x7f891c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f891c06ae40 0x7f891c06d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8924000f60 tx=0x7f8924009040 comp rx=0 tx=0).stop 2023-11-24T06:29:42.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 msgr2=0x7f893c0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f893c0ca160 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f892c037040 tx=0x7f892c004730 comp rx=0 tx=0).stop 2023-11-24T06:29:42.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 shutdown_connections 2023-11-24T06:29:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f891c06ae40 0x7f891c06d300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f893c0cada0 0x7f893c0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893c153140 0x7f893c154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 --2- 172.21.15.64:0/4005636290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89280039a0 0x7f893c0ca160 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 >> 172.21.15.64:0/4005636290 conn(0x7f893c0d3090 msgr2=0x7f893c06aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 shutdown_connections 2023-11-24T06:29:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:41.556+0000 7f8940973640 1 -- 172.21.15.64:0/4005636290 wait complete. 2023-11-24T06:29:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2023-11-24T06:29:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2023-11-24T06:29:42.889 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.893 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2023-11-24T06:29:42.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2023-11-24T06:29:42.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.898 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.902 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.905 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.907 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.912 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.915 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.916 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.917 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.921 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2023-11-24T06:29:42.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.926 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.927 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.929 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.929 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.929 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.929 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.930 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.934 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.936 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.940 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.942 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2023-11-24T06:29:42.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2023-11-24T06:29:42.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-24T06:29:42.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.953 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2023-11-24T06:29:42.954 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.955 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2023-11-24T06:29:42.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.962 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.966 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.967 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2023-11-24T06:29:42.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-24T06:29:42.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.976 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2023-11-24T06:29:42.984 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.985 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.986 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.987 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:42.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:42.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:42.989 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:42.990 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.993 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2023-11-24T06:29:42.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2023-11-24T06:29:42.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.996 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:42.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:42.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-11-24T06:29:42.999 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2023-11-24T06:29:43.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.001 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-11-24T06:29:43.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:43.006 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-11-24T06:29:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-11-24T06:29:43.010 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-11-24T06:29:43.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.015 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.016 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-11-24T06:29:43.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-11-24T06:29:43.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: "services": {}, 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: "always_on_modules": { 2023-11-24T06:29:43.019 INFO:teuthology.orchestra.run.smithi064.stdout: "octopus": [ 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-11-24T06:29:43.020 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "pacific": [ 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-11-24T06:29:43.021 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: "quincy": [ 2023-11-24T06:29:43.022 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-11-24T06:29:43.023 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: "reef": [ 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-11-24T06:29:43.024 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.025 INFO:teuthology.orchestra.run.smithi064.stdout: "last_failure_osd_epoch": 0, 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: "active_clients": [ 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-11-24T06:29:43.026 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 853547399 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "libcephsqlite", 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3221935172 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-11-24T06:29:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2415678094 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 773347189 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:43.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.031 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T06:29:43.031 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:43.035 DEBUG:tasks.check_counter:Getting stats from y 2023-11-24T06:29:43.035 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mgr.y.asok perf dump 2023-11-24T06:29:43.262 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:43.263 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-11-24T06:29:43.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 409, 2023-11-24T06:29:43.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 221, 2023-11-24T06:29:43.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2992896, 2023-11-24T06:29:43.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 3063971, 2023-11-24T06:29:43.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 110, 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 13, 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.210784191, 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.019517926, 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.154043252, 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.008075651, 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 212, 2023-11-24T06:29:43.264 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.048452540, 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000228549 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.265 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2992896, 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 3063840 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 283, 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 49, 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2430770, 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 50355, 2023-11-24T06:29:43.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 49, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 12, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.083683692, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.004098453, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.100901889, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 33, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.035592810, 2023-11-24T06:29:43.267 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001078570 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2429888, 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 49952 2023-11-24T06:29:43.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 646, 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 415, 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2896496, 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 4729155, 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 162, 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 22, 2023-11-24T06:29:43.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.263901171, 2023-11-24T06:29:43.270 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.067745204, 2023-11-24T06:29:43.270 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.229808312, 2023-11-24T06:29:43.270 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000024816, 2023-11-24T06:29:43.270 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-11-24T06:29:43.270 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 400, 2023-11-24T06:29:43.270 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.062685044, 2023-11-24T06:29:43.270 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000156712 2023-11-24T06:29:43.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.271 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-11-24T06:29:43.271 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.271 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.271 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2896496, 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 4729024 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: "client": { 2023-11-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": { 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: "lat": { 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: "wrlat": { 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: "rdlat": { 2023-11-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.275 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.275 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.275 INFO:teuthology.orchestra.run.smithi064.stdout: "fsync": { 2023-11-24T06:29:43.276 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.276 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.276 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.276 INFO:teuthology.orchestra.run.smithi064.stdout: "mdavg": 0.000000000, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsqsum": 0, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "mdops": 0, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "readavg": 0.000000000, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "readsqsum": 0, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "rdops": 0, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "writeavg": 0.000000000, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "writesqsum": 0, 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: "rdops": 0 2023-11-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-Mgr": { 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2, 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.059657978, 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.029828989 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MgrStandby": { 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-balancer": { 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-cmd_finisher": { 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 16, 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.001343704, 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000083981 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-crash": { 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-devicehealth": { 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-iostat": { 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-nfs": { 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-orchestrator": { 2023-11-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.289 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-pg_autoscaler": { 2023-11-24T06:29:43.289 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.289 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.289 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.289 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-progress": { 2023-11-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-rbd_support": { 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.291 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-restful": { 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 16, 2023-11-24T06:29:43.292 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.006035237, 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000377202 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-status": { 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.293 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-telemetry": { 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.294 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-volumes": { 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.001093149, 2023-11-24T06:29:43.295 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001093149 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: "libcephsqlite_striper": { 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: "update_metadata": 8, 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: "update_allocated": 2, 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: "update_size": 6, 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: "update_version": 8, 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: "shrink": 0, 2023-11-24T06:29:43.296 INFO:teuthology.orchestra.run.smithi064.stdout: "shrink_bytes": 0, 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: "lock": 11, 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: "unlock": 11 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: "libcephsqlite_vfs": { 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: "op_open": { 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2, 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.043999123, 2023-11-24T06:29:43.297 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.021999561 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: "op_delete": { 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: "op_access": { 2023-11-24T06:29:43.298 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: "op_fullpathname": { 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000999980, 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000999980 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.299 INFO:teuthology.orchestra.run.smithi064.stdout: "op_currenttime": { 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4, 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_close": { 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.300 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_read": { 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 20, 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.023999524, 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001199976 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_write": { 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 85, 2023-11-24T06:29:43.301 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013999721, 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000164702 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_truncate": { 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.302 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_sync": { 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 24, 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.222995562, 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.009291481 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_filesize": { 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 20, 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_lock": { 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 27, 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.060998785, 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002259214 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_unlock": { 2023-11-24T06:29:43.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 19, 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.043999123, 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002315743 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_checkreservedlock": { 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 9, 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.305 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_filecontrol": { 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 26, 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000999980, 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000038460 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_sectorsize": { 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.306 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "opf_devicecharacteristics": { 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 30, 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-24T06:29:43.308 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-11-24T06:29:43.309 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-11-24T06:29:43.310 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-11-24T06:29:43.311 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 1611800, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 271, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 1496, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 17, 2023-11-24T06:29:43.312 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 33112, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 600, 2023-11-24T06:29:43.313 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 12880, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 308, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 0, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 0, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-11-24T06:29:43.314 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: "cache_hit": 0, 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: "cache_miss": 0 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: "objectcacher-libcephfs": { 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: "cache_ops_hit": 0, 2023-11-24T06:29:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: "cache_ops_miss": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "cache_bytes_hit": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "cache_bytes_miss": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "data_read": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "data_written": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "data_flushed": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "data_overwritten_while_flushing": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "write_ops_blocked": 0, 2023-11-24T06:29:43.316 INFO:teuthology.orchestra.run.smithi064.stdout: "write_bytes_blocked": 0, 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "write_time_blocked": 0.000000000 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 0, 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 0, 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 0, 2023-11-24T06:29:43.317 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 0, 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-11-24T06:29:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 0, 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 0, 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-11-24T06:29:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2023-11-24T06:29:43.320 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-11-24T06:29:43.321 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-11-24T06:29:43.322 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-11-24T06:29:43.323 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 8, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 39, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-11-24T06:29:43.324 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 38, 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 8, 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 8, 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2023-11-24T06:29:43.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter-0x56477cdb46e0": { 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 163, 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 2434716, 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 0, 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 163, 2023-11-24T06:29:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 163, 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.205975990, 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.007398625 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 163, 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 307 2023-11-24T06:29:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 163, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 46, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 117, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 2, 2023-11-24T06:29:43.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 20, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 85, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-11-24T06:29:43.329 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 104, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 52, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 22, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 22, 2023-11-24T06:29:43.330 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-11-24T06:29:43.331 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-11-24T06:29:43.332 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 19, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 9, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 1, 2023-11-24T06:29:43.333 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 1, 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter-0x56477cdb4b40": { 2023-11-24T06:29:43.334 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 0, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 0, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 0, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 0, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.335 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.336 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-11-24T06:29:43.336 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-11-24T06:29:43.336 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.336 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0 2023-11-24T06:29:43.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.336 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 0, 2023-11-24T06:29:43.337 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-11-24T06:29:43.338 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-11-24T06:29:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 0, 2023-11-24T06:29:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2023-11-24T06:29:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-11-24T06:29:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-11-24T06:29:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-11-24T06:29:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-11-24T06:29:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-11-24T06:29:43.340 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-11-24T06:29:43.341 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 0, 2023-11-24T06:29:43.342 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 0, 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 0, 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter-0x56477cdb4dc0": { 2023-11-24T06:29:43.343 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 0, 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 0, 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 0, 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 0, 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.344 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 0, 2023-11-24T06:29:43.345 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2023-11-24T06:29:43.346 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-11-24T06:29:43.347 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-11-24T06:29:43.348 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-11-24T06:29:43.349 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-11-24T06:29:43.350 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 0, 2023-11-24T06:29:43.351 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter-0x56477d8dc000": { 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-11-24T06:29:43.352 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 1, 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 39, 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 0, 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 1, 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.003999921, 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003999921 2023-11-24T06:29:43.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 1, 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 1, 2023-11-24T06:29:43.354 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2023-11-24T06:29:43.355 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 0, 2023-11-24T06:29:43.356 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T06:29:43.357 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 1, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 0, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 0, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 1, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 1, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_client_bytes": { 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 134217728, 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 65, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 166403, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 65, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 65, 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 166403, 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_client_messages": { 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 512, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 65, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 65, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 65, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.364 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 65, 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 65, 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_mds_bytes": { 2023-11-24T06:29:43.365 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 134217728, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 118, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 823768, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 118, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 118, 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 823768, 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_mds_messsages": { 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 128, 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 118, 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 118, 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 118, 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 118, 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 118, 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_mon_bytes": { 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 134217728, 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 87, 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 533606, 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 87, 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 87, 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 533606, 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.371 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_mon_messsages": { 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 128, 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 87, 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 87, 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 87, 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 87, 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 87, 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_osd_bytes": { 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 536870912, 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 454, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 4755571, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 454, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 454, 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 4755571, 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-mgr_osd_messsages": { 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 8192, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 454, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 454, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 454, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 454, 2023-11-24T06:29:43.378 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 454, 2023-11-24T06:29:43.378 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.378 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.378 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.378 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mgr": { 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 388, 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 682525, 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.379 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 388, 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 388, 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 682525, 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.380 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mgr-0x564776d17680": { 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 724, 2023-11-24T06:29:43.381 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 6279348, 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 724, 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 724, 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 6279348, 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.382 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-radosclient": { 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.383 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 176, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 359564, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 176, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 176, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 359564, 2023-11-24T06:29:43.384 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x56477cdb4d20": { 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 9, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 314711, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 9, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.386 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 9, 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 314711, 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x56477cdb4fa0": { 2023-11-24T06:29:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 10, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 314735, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 10, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.388 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 10, 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 314735, 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x56477cdb5540": { 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 8, 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 184480, 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.390 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 8, 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 8, 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 184480, 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.391 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-11-24T06:29:43.392 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.393 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.394 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.394 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.394 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes-0x56477cdb4640": { 2023-11-24T06:29:43.394 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.394 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 163, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 2433027, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 163, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 163, 2023-11-24T06:29:43.395 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 2433027, 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes-0x56477cdb4c80": { 2023-11-24T06:29:43.396 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.397 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.398 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes-0x56477cdb4f00": { 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-11-24T06:29:43.399 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.400 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes-0x56477cdb55e0": { 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 1, 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.401 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 1, 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1, 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.402 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-11-24T06:29:43.403 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops-0x56477cdb45a0": { 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-11-24T06:29:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 163, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 163, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 163, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 163, 2023-11-24T06:29:43.406 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 163, 2023-11-24T06:29:43.407 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.407 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.407 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.407 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.407 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.407 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops-0x56477cdb4be0": { 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-11-24T06:29:43.408 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.409 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops-0x56477cdb4e60": { 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-11-24T06:29:43.410 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.411 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.412 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.412 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T06:29:43.412 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops-0x56477cdb54a0": { 2023-11-24T06:29:43.412 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-11-24T06:29:43.412 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-11-24T06:29:43.412 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 1, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 1, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1, 2023-11-24T06:29:43.413 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1, 2023-11-24T06:29:43.414 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-11-24T06:29:43.414 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-11-24T06:29:43.414 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-11-24T06:29:43.414 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-11-24T06:29:43.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T06:29:43.414 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:43.416 INFO:tasks.check_counter:Daemon mgr.y finisher-volumes.complete_latency.avgcount=1 2023-11-24T06:29:43.416 INFO:tasks.check_counter:Daemon mgr.y finisher-volumes.queue_len=0 2023-11-24T06:29:43.416 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mgr daemons: {'finisher-volumes.complete_latency.avgcount'} 2023-11-24T06:29:43.421 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-24T06:29:43.431 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-24T06:29:43.431 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:29:43.431 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:43.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:29:43.498 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T06:29:43.498 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:43.556 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:29:43.556 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:29:43.556 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:43.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:29:43.619 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-11-24T06:29:43.619 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:43.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:29:43.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:43.675 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T06:29:43.738 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:29:43.738 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:29:43.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:43.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:43.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:43.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:43.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:43.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-11-24T06:29:44.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:44.097 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-11-24T06:29:44.158 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:29:44.159 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:29:44.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:44.160 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:44.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:44.259 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:44.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:44.354 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2023-11-24T06:29:44.486 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T06:29:44.496 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:29:44.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:44.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.731+0000 7fb13759e640 1 Processor -- start 2023-11-24T06:29:44.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.731+0000 7fb13759e640 1 -- start start 2023-11-24T06:29:44.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.732+0000 7fb13759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb1380b3eb0 0x7fb1380b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.732+0000 7fb13759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb138161ee0 con 0x7fb1381607a0 2023-11-24T06:29:44.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.732+0000 7fb13759e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1380b2b90 con 0x7fb138160b80 2023-11-24T06:29:44.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.732+0000 7fb13759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1380d21d0 con 0x7fb1380b3eb0 2023-11-24T06:29:44.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.733+0000 7fb135d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb1380b3eb0 0x7fb1380b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.733+0000 7fb135d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb1380b3eb0 0x7fb1380b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45896/0 (socket says 172.21.15.64:45896) 2023-11-24T06:29:44.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.733+0000 7fb135d9b640 1 -- 172.21.15.64:0/4244103929 learned_addr learned my addr 172.21.15.64:0/4244103929 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:44.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.733+0000 7fb135d9b640 1 -- 172.21.15.64:0/4244103929 >> v1:172.21.15.162:6789/0 conn(0x7fb138160b80 legacy=0x7fb1381617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:44.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.733+0000 7fb135d9b640 1 -- 172.21.15.64:0/4244103929 >> v1:172.21.15.64:6789/0 conn(0x7fb1381607a0 legacy=0x7fb1381610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.733+0000 7fb135d9b640 1 -- 172.21.15.64:0/4244103929 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1380b2e30 con 0x7fb1380b3eb0 2023-11-24T06:29:44.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.735+0000 7fb135d9b640 1 --2- 172.21.15.64:0/4244103929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1380b3eb0 0x7fb1380b4330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb120009870 tx=0x7fb12002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c0c886fa58ce14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:44.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.735+0000 7fb13559a640 1 -- 172.21.15.64:0/4244103929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120030850 con 0x7fb1380b3eb0 2023-11-24T06:29:44.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.735+0000 7fb13559a640 1 -- 172.21.15.64:0/4244103929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1200309b0 con 0x7fb1380b3eb0 2023-11-24T06:29:44.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.735+0000 7fb13559a640 1 -- 172.21.15.64:0/4244103929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120030d40 con 0x7fb1380b3eb0 2023-11-24T06:29:44.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.736+0000 7fb13759e640 1 -- 172.21.15.64:0/4244103929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1380b3eb0 msgr2=0x7fb1380b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.736+0000 7fb13759e640 1 --2- 172.21.15.64:0/4244103929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1380b3eb0 0x7fb1380b4330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb120009870 tx=0x7fb12002b990 comp rx=0 tx=0).stop 2023-11-24T06:29:44.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.736+0000 7fb13759e640 1 -- 172.21.15.64:0/4244103929 shutdown_connections 2023-11-24T06:29:44.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.736+0000 7fb13759e640 1 --2- 172.21.15.64:0/4244103929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1380b3eb0 0x7fb1380b4330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.736+0000 7fb13759e640 1 -- 172.21.15.64:0/4244103929 >> 172.21.15.64:0/4244103929 conn(0x7fb13809c590 msgr2=0x7fb1380d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:44.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.736+0000 7fb13759e640 1 -- 172.21.15.64:0/4244103929 shutdown_connections 2023-11-24T06:29:44.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.736+0000 7fb13759e640 1 -- 172.21.15.64:0/4244103929 wait complete. 2023-11-24T06:29:44.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.739+0000 7fb13759e640 1 Processor -- start 2023-11-24T06:29:44.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.739+0000 7fb13759e640 1 -- start start 2023-11-24T06:29:44.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb13759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1380b3eb0 0x7fb1380ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb13759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1381607a0 0x7fb13815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb13759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb138160b80 0x7fb1381512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb13759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380cc6b0 con 0x7fb1381607a0 2023-11-24T06:29:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb13759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380cd010 con 0x7fb138160b80 2023-11-24T06:29:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb13759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380cba30 con 0x7fb1380b3eb0 2023-11-24T06:29:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb135d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1381607a0 0x7fb13815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb135d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1381607a0 0x7fb13815d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45906/0 (socket says 172.21.15.64:45906) 2023-11-24T06:29:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.740+0000 7fb135d9b640 1 -- 172.21.15.64:0/1361477923 learned_addr learned my addr 172.21.15.64:0/1361477923 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb135d9b640 1 -- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1380b3eb0 msgr2=0x7fb1380ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb136d9d640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb138160b80 0x7fb1381512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb13659c640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1380b3eb0 0x7fb1380ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb135d9b640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1380b3eb0 0x7fb1380ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb135d9b640 1 -- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb138160b80 msgr2=0x7fb1381512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb135d9b640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb138160b80 0x7fb1381512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb135d9b640 1 -- 172.21.15.64:0/1361477923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1200094d0 con 0x7fb1381607a0 2023-11-24T06:29:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb136d9d640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb138160b80 0x7fb1381512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb13659c640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1380b3eb0 0x7fb1380ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.741+0000 7fb135d9b640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1381607a0 0x7fb13815d6a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb120009870 tx=0x7fb120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.742+0000 7fb12effd640 1 -- 172.21.15.64:0/1361477923 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120030800 con 0x7fb1381607a0 2023-11-24T06:29:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.742+0000 7fb12effd640 1 -- 172.21.15.64:0/1361477923 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb120039d90 con 0x7fb1381607a0 2023-11-24T06:29:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.742+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1380cbd10 con 0x7fb1381607a0 2023-11-24T06:29:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.742+0000 7fb12effd640 1 -- 172.21.15.64:0/1361477923 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120043630 con 0x7fb1381607a0 2023-11-24T06:29:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.742+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1380cc2d0 con 0x7fb1381607a0 2023-11-24T06:29:44.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.743+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1380cc7f0 con 0x7fb1381607a0 2023-11-24T06:29:44.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.744+0000 7fb12effd640 1 -- 172.21.15.64:0/1361477923 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb12002e070 con 0x7fb1381607a0 2023-11-24T06:29:44.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.745+0000 7fb12effd640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb11806ae70 0x7fb11806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.745+0000 7fb12effd640 1 -- 172.21.15.64:0/1361477923 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb1200ab790 con 0x7fb1381607a0 2023-11-24T06:29:44.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.745+0000 7fb13659c640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb11806ae70 0x7fb11806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.746+0000 7fb13659c640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb11806ae70 0x7fb11806d330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb1280012e0 tx=0x7fb128009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:44.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.750+0000 7fb12effd640 1 -- 172.21.15.64:0/1361477923 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1380cc7f0 con 0x7fb1381607a0 2023-11-24T06:29:44.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.940+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1380b36d0 con 0x7fb11806ae70 2023-11-24T06:29:44.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.946+0000 7fb12effd640 1 -- 172.21.15.64:0/1361477923 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48276 (secure 0 0 0) 0x7fb12800d950 con 0x7fb11806ae70 2023-11-24T06:29:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:44.950 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:29:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb11806ae70 msgr2=0x7fb11806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb11806ae70 0x7fb11806d330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb1280012e0 tx=0x7fb128009040 comp rx=0 tx=0).stop 2023-11-24T06:29:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1381607a0 msgr2=0x7fb13815d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1381607a0 0x7fb13815d6a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb120009870 tx=0x7fb120004730 comp rx=0 tx=0).stop 2023-11-24T06:29:44.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 shutdown_connections 2023-11-24T06:29:44.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fb11806ae70 0x7fb11806d330 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb138160b80 0x7fb1381512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1381607a0 0x7fb13815d6a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 --2- 172.21.15.64:0/1361477923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1380b3eb0 0x7fb1380ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 >> 172.21.15.64:0/1361477923 conn(0x7fb13809c590 msgr2=0x7fb13815e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:44.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 shutdown_connections 2023-11-24T06:29:44.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:44.949+0000 7fb13759e640 1 -- 172.21.15.64:0/1361477923 wait complete. 2023-11-24T06:29:44.970 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-11-24T06:29:43.137707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220736,"kb_used_data":4552,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753984,"statfs":{"total":773094113280,"available":772868079616,"internally_reserved":0,"allocated":4661248,"data_stored":2554696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4388,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.229916"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-24T06:29:39.128458+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:29:39.128458+0000","last_peered":"2023-11-24T06:29:39.128458+0000","last_clean":"2023-11-24T06:29:39.128458+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:29:39.128458+0000","last_undegraded":"2023-11-24T06:29:39.128458+0000","last_fullsized":"2023-11-24T06:29:39.128458+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1028096,"data_stored":762113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97399999999999998,"15min":0.97699999999999998},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.633,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0469999999999999,"15min":1.0589999999999999},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7669999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.82599999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0529999999999999,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.4930000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.141,"15min":1.169},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.581,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0960000000000001,"15min":1.091},"min":{"1min":0.64100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.216},{"interface":"front","average":{"1min":1.127,"5min":1.1779999999999999,"15min":1.1859999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.145,"15min":1.149},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.347,"15min":1.369},"min":{"1min":0.65300000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.871,"5min":2.024,"15min":2.024},"last":1.6399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.758,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3380000000000001,"15min":1.343},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.5940000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.379,"15min":1.383},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.093,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.766},{"interface":"front","average":{"1min":1.373,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.56599999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.853,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.1819999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.476,"15min":1.468},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.29},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.397,"15min":1.393},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4199999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":171745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.526,"5min":2.323,"15min":2.323},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97299999999999998,"15min":0.96999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.0700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5669999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":1.679}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.048,"15min":1.0329999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.131,"15min":1.1259999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.151,"15min":1.167},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2290000000000001,"15min":1.2470000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.0389999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.288,"15min":1.302},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.79},{"interface":"front","average":{"1min":1.3,"5min":1.379,"15min":1.3919999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.847,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.458}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.425,"15min":1.446},"min":{"1min":0.77900000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.887,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.887},{"interface":"front","average":{"1min":1.296,"5min":1.417,"15min":1.4370000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8440000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.6399999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5289999999999999,"15min":1.5549999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.839,"5min":3.052,"15min":3.052},"last":1.742},{"interface":"front","average":{"1min":1.254,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.732,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.2110000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":171745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.53100000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.1120000000000001,"15min":1.1279999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.671,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.671}]},{"osd":1,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.105,"15min":1.1140000000000001},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.6439999999999999,"5min":1.79,"15min":1.79},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.224,"15min":1.24},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8320000000000001,"5min":1.857,"15min":1.857},"last":1.7090000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.222,"15min":1.24},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.897,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.2589999999999999,"15min":1.2849999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6499999999999999,"5min":1.986,"15min":1.986},"last":1.2949999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.849,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.849},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.329,"15min":1.361},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.746,"5min":2.359,"15min":2.359},"last":1.478}]},{"osd":4,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.3260000000000001,"15min":1.3580000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.702,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.377,"15min":1.385},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1160000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":2.1160000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.4299999999999999,"15min":1.46},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9630000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.9179999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.464,"15min":1.488},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9610000000000001,"5min":3.0299999999999998,"15min":3.0299999999999998},"last":1.6279999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5249999999999999,"15min":1.5389999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2570000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":2.2570000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.5880000000000001,"15min":1.6220000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1970000000000001,"5min":2.552,"15min":2.552},"last":2.012}]}]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.113,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2210000000000001,"15min":1.2370000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.3149999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64600000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.8879999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1459999999999999,"15min":1.1479999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.978,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.79700000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6870000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.244},"min":{"1min":0.69999999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0840000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.234}]},{"osd":3,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.371},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2480000000000002,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.272},{"interface":"front","average":{"1min":1.111,"5min":1.2130000000000001,"15min":1.23},"min":{"1min":0.58399999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0659999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.86399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.454,"15min":1.474},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2400000000000002,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.51,"15min":1.53},"min":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.157,"5min":4.2060000000000004,"15min":4.2060000000000004},"last":1.1899999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.381,"15min":1.401},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8220000000000001,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5580000000000001,"15min":1.577},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.3029999999999999,"5min":4.335,"15min":4.335},"last":1.0329999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":3.7959999999999998,"5min":2.0779999999999998,"15min":1.792},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":40.311999999999998,"5min":40.311999999999998,"15min":40.311999999999998},"last":1.399},{"interface":"front","average":{"1min":3.7799999999999998,"5min":2.0840000000000001,"15min":1.802},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":40.286000000000001,"5min":40.286000000000001,"15min":40.286000000000001},"last":1.3600000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.5720000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.2190000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.86299999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.73199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.47}]},{"osd":5,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.5129999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.123},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.587}]},{"osd":7,"last update":"Fri Nov 24 06:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.3859999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.075},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91600000000000004,"15min":0.93600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.5469999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.94799999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.024},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3169999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.044,"15min":1.0780000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.569,"5min":2.278,"15min":2.278},"last":1.2709999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.353,"5min":1.847,"15min":1.847},"last":1.353},{"interface":"front","average":{"1min":1.032,"5min":1.232,"15min":1.2649999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7130000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.4079999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.323,"15min":1.345},"min":{"1min":0.69099999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7450000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.669},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.2090000000000001,"15min":1.2350000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.712,"5min":3.3839999999999999,"15min":3.3839999999999999},"last":1.712}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.345,"15min":1.361},"min":{"1min":0.87,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.821,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.536},{"interface":"front","average":{"1min":1.153,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8480000000000001,"5min":3.706,"15min":3.706},"last":1.452}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2869999999999999,"15min":1.319},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7729999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.613},{"interface":"front","average":{"1min":1.288,"5min":1.4019999999999999,"15min":1.421},"min":{"1min":0.872,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.913,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4870000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.46,"15min":1.5029999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.99,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.4119999999999999,"15min":1.4590000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.9910000000000001,"5min":3.661,"15min":3.661},"last":1.504}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96499999999999997,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3160000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.016,"15min":1.026},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.149}]},{"osd":2,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94499999999999995,"15min":0.96199999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.506},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.034,"15min":1.052},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.534,"5min":1.998,"15min":1.998},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.1140000000000001,"15min":1.137},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7589999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.079,"15min":1.087},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6870000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.363}]},{"osd":4,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.226,"15min":1.26},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4650000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.204},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3720000000000001,"15min":1.3759999999999999},"min":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.905,"5min":2.415,"15min":2.415},"last":1.4399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.365,"15min":1.3979999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.8360000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.073}]},{"osd":6,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.47,"15min":1.48},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1259999999999999,"5min":2.544,"15min":2.544},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.4630000000000001,"15min":1.4890000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.909,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.712}]},{"osd":7,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.488,"15min":1.504},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0270000000000001,"5min":2.355,"15min":2.355},"last":1.671},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5329999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.056,"5min":2.492,"15min":2.492},"last":1.6160000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1028096,"data_stored":762113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.94699999999999995,"15min":0.98699999999999999},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.133,"5min":2.54,"15min":2.54},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.95199999999999996,"15min":0.98799999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.599,"5min":1.607,"15min":1.607},"last":0.504}]},{"osd":2,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.0840000000000001,"15min":1.1319999999999999},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.2609999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.91200000000000003,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.107,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.123},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4410000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.173,"15min":1.206},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5700000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.83799999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3660000000000001,"15min":1.403},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5209999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.212},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.494,"15min":1.542},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5209999999999999,"5min":2.456,"15min":2.456},"last":1.1719999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3999999999999999,"15min":1.427},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.4390000000000001,"15min":1.5069999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.587,"15min":1.665},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7470000000000001,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.258,"5min":1.5429999999999999,"15min":1.591},"min":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.8879999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.1379999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.4510000000000001,"15min":1.5089999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7929999999999999,"5min":2.802,"15min":2.802},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.631,"15min":1.698},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.8859999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.246}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:44.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.187+0000 7f4d5529f640 1 Processor -- start 2023-11-24T06:29:45.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.187+0000 7f4d5529f640 1 -- start start 2023-11-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d5529f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d50159590 0x7f4d50070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d5529f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d500b3c20 0x7f4d500ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d5529f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d50071790 con 0x7f4d500cd220 2023-11-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d5529f640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d50071100 con 0x7f4d500b3c20 2023-11-24T06:29:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d5529f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d50070860 con 0x7f4d50159590 2023-11-24T06:29:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d4f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d500b3c20 0x7f4d500ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d4f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d500b3c20 0x7f4d500ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45916/0 (socket says 172.21.15.64:45916) 2023-11-24T06:29:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/911703616 learned_addr learned my addr 172.21.15.64:0/911703616 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.189+0000 7f4d54a9e640 1 --1- 172.21.15.64:0/911703616 >> v1:172.21.15.64:6789/0 conn(0x7f4d500cd220 0x7f4d5006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35852/0 (socket says 172.21.15.64:35852) 2023-11-24T06:29:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.190+0000 7f4d4ffff640 1 --2- 172.21.15.64:0/911703616 >> v2:172.21.15.162:3300/0 conn(0x7f4d50159590 0x7f4d50070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.190+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/911703616 >> v2:172.21.15.162:3300/0 conn(0x7f4d50159590 msgr2=0x7f4d50070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.190+0000 7f4d4f7fe640 1 --2- 172.21.15.64:0/911703616 >> v2:172.21.15.162:3300/0 conn(0x7f4d50159590 0x7f4d50070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.190+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/911703616 >> v1:172.21.15.64:6789/0 conn(0x7f4d500cd220 legacy=0x7f4d5006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.190+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/911703616 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50070b40 con 0x7f4d500b3c20 2023-11-24T06:29:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.191+0000 7f4d4f7fe640 1 --2- 172.21.15.64:0/911703616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500b3c20 0x7f4d500ccce0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d500b30a0 tx=0x7f4d4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be61a2bcfe97995 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.191+0000 7f4d4effd640 1 -- 172.21.15.64:0/911703616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4002ca80 con 0x7f4d500b3c20 2023-11-24T06:29:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.191+0000 7f4d4effd640 1 -- 172.21.15.64:0/911703616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4002cbe0 con 0x7f4d500b3c20 2023-11-24T06:29:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.191+0000 7f4d4effd640 1 -- 172.21.15.64:0/911703616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d400343f0 con 0x7f4d500b3c20 2023-11-24T06:29:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 -- 172.21.15.64:0/911703616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500b3c20 msgr2=0x7f4d500ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 --2- 172.21.15.64:0/911703616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500b3c20 0x7f4d500ccce0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d500b30a0 tx=0x7f4d4002bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 -- 172.21.15.64:0/911703616 shutdown_connections 2023-11-24T06:29:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 --2- 172.21.15.64:0/911703616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500b3c20 0x7f4d500ccce0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 --2- 172.21.15.64:0/911703616 >> v2:172.21.15.162:3300/0 conn(0x7f4d50159590 0x7f4d50070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 -- 172.21.15.64:0/911703616 >> 172.21.15.64:0/911703616 conn(0x7f4d5006a800 msgr2=0x7f4d5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 -- 172.21.15.64:0/911703616 shutdown_connections 2023-11-24T06:29:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.192+0000 7f4d5529f640 1 -- 172.21.15.64:0/911703616 wait complete. 2023-11-24T06:29:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.194+0000 7f4d5529f640 1 Processor -- start 2023-11-24T06:29:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.194+0000 7f4d5529f640 1 -- start start 2023-11-24T06:29:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.195+0000 7f4d5529f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d500b3c20 0x7f4d50140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.195+0000 7f4d5529f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500cd220 0x7f4d50152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.195+0000 7f4d5529f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d50159590 0x7f4d50152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.195+0000 7f4d5529f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50144610 con 0x7f4d500cd220 2023-11-24T06:29:45.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.195+0000 7f4d5529f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5015b710 con 0x7f4d50159590 2023-11-24T06:29:45.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.195+0000 7f4d5529f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5015ab90 con 0x7f4d500b3c20 2023-11-24T06:29:45.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.195+0000 7f4d4f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500cd220 0x7f4d50152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500cd220 0x7f4d50152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45930/0 (socket says 172.21.15.64:45930) 2023-11-24T06:29:45.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/4204780781 learned_addr learned my addr 172.21.15.64:0/4204780781 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:45.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4ffff640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d500b3c20 0x7f4d50140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/4204780781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d500b3c20 msgr2=0x7f4d50140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d500b3c20 0x7f4d50140870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/4204780781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d50159590 msgr2=0x7f4d50152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d50159590 0x7f4d50152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 -- 172.21.15.64:0/4204780781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40009b30 con 0x7f4d500cd220 2023-11-24T06:29:45.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.196+0000 7f4d4f7fe640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500cd220 0x7f4d50152aa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4d40000f80 tx=0x7f4d40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.197+0000 7f4d4cff9640 1 -- 172.21.15.64:0/4204780781 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d40009510 con 0x7f4d500cd220 2023-11-24T06:29:45.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.197+0000 7f4d4cff9640 1 -- 172.21.15.64:0/4204780781 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4003e450 con 0x7f4d500cd220 2023-11-24T06:29:45.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.197+0000 7f4d4cff9640 1 -- 172.21.15.64:0/4204780781 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4003e800 con 0x7f4d500cd220 2023-11-24T06:29:45.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.197+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5015ae70 con 0x7f4d500cd220 2023-11-24T06:29:45.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.197+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5015b430 con 0x7f4d500cd220 2023-11-24T06:29:45.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.198+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d50144780 con 0x7f4d500cd220 2023-11-24T06:29:45.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.199+0000 7f4d4cff9640 1 -- 172.21.15.64:0/4204780781 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d4002cd50 con 0x7f4d500cd220 2023-11-24T06:29:45.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.200+0000 7f4d4cff9640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d3006ae40 0x7f4d3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.200+0000 7f4d4cff9640 1 -- 172.21.15.64:0/4204780781 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4d400abe30 con 0x7f4d500cd220 2023-11-24T06:29:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.200+0000 7f4d4ffff640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d3006ae40 0x7f4d3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.201+0000 7f4d4ffff640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d3006ae40 0x7f4d3006d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d38006fd0 tx=0x7f4d38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.205+0000 7f4d4cff9640 1 -- 172.21.15.64:0/4204780781 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d50144780 con 0x7f4d500cd220 2023-11-24T06:29:45.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.397+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d500b1f30 con 0x7f4d3006ae40 2023-11-24T06:29:45.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.401+0000 7f4d4cff9640 1 -- 172.21.15.64:0/4204780781 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48335 (secure 0 0 0) 0x7f4d3800e4b0 con 0x7f4d3006ae40 2023-11-24T06:29:45.404 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:45.404 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:29:45.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.403+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d3006ae40 msgr2=0x7f4d3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d3006ae40 0x7f4d3006d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d38006fd0 tx=0x7f4d38008040 comp rx=0 tx=0).stop 2023-11-24T06:29:45.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500cd220 msgr2=0x7f4d50152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500cd220 0x7f4d50152aa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4d40000f80 tx=0x7f4d40004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:45.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 shutdown_connections 2023-11-24T06:29:45.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d3006ae40 0x7f4d3006d300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d50159590 0x7f4d50152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d500cd220 0x7f4d50152aa0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 --2- 172.21.15.64:0/4204780781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d500b3c20 0x7f4d50140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 >> 172.21.15.64:0/4204780781 conn(0x7f4d5006a800 msgr2=0x7f4d500b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:45.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 shutdown_connections 2023-11-24T06:29:45.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.404+0000 7f4d5529f640 1 -- 172.21.15.64:0/4204780781 wait complete. 2023-11-24T06:29:45.425 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-11-24T06:29:45.138242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220856,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753864,"statfs":{"total":773094113280,"available":772867956736,"internally_reserved":0,"allocated":4792320,"data_stored":2599216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4388,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002865"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-24T06:29:39.128458+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:29:39.128458+0000","last_peered":"2023-11-24T06:29:39.128458+0000","last_clean":"2023-11-24T06:29:39.128458+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:29:39.128458+0000","last_undegraded":"2023-11-24T06:29:39.128458+0000","last_fullsized":"2023-11-24T06:29:39.128458+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97399999999999998,"15min":0.97699999999999998},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.633,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0469999999999999,"15min":1.0589999999999999},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7669999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.292}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0529999999999999,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.3839999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.141,"15min":1.169},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.581,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.3680000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0960000000000001,"15min":1.091},"min":{"1min":0.64100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.1779999999999999,"15min":1.1859999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.995}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.145,"15min":1.149},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.347,"15min":1.369},"min":{"1min":0.65300000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.871,"5min":2.024,"15min":2.024},"last":1.139}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.758,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3380000000000001,"15min":1.343},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.379,"15min":1.383},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.093,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.254},{"interface":"front","average":{"1min":1.373,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.56599999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.853,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.4650000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.476,"15min":1.468},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.397,"15min":1.393},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.506}]}]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.526,"5min":2.323,"15min":2.323},"last":1.73},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97299999999999998,"15min":0.96999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2889999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.0700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5669999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":1.597}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.048,"15min":1.0329999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.131,"15min":1.1259999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.3160000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.151,"15min":1.167},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2290000000000001,"15min":1.2470000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.2410000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.288,"15min":1.302},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.379,"15min":1.3919999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.847,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.95399999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.425,"15min":1.446},"min":{"1min":0.77900000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.887,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.417,"15min":1.4370000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8440000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5289999999999999,"15min":1.5549999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.839,"5min":3.052,"15min":3.052},"last":1.347},{"interface":"front","average":{"1min":1.254,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.732,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.111}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.53100000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.413},{"interface":"front","average":{"1min":1.016,"5min":1.1120000000000001,"15min":1.1279999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.671,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.361}]},{"osd":1,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.105,"15min":1.1140000000000001},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.6439999999999999,"5min":1.79,"15min":1.79},"last":0.871},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.224,"15min":1.24},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8320000000000001,"5min":1.857,"15min":1.857},"last":1.5029999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.222,"15min":1.24},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.897,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.2589999999999999,"15min":1.2849999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6499999999999999,"5min":1.986,"15min":1.986},"last":1.1759999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.849,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.329,"15min":1.361},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.746,"5min":2.359,"15min":2.359},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.3260000000000001,"15min":1.3580000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.702,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.377,"15min":1.385},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1160000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.6459999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.4299999999999999,"15min":1.46},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9630000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.464,"15min":1.488},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9610000000000001,"5min":3.0299999999999998,"15min":3.0299999999999998},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5249999999999999,"15min":1.5389999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2570000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.5880000000000001,"15min":1.6220000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1970000000000001,"5min":2.552,"15min":2.552},"last":1.6140000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.113,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2210000000000001,"15min":1.2370000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64600000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.8879999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1459999999999999,"15min":1.1479999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.978,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.131}]},{"osd":2,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6870000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.244},"min":{"1min":0.69999999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0840000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.371},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2480000000000002,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.2130000000000001,"15min":1.23},"min":{"1min":0.58399999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0659999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.304}]},{"osd":5,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.454,"15min":1.474},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2400000000000002,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.51,"15min":1.53},"min":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.157,"5min":4.2060000000000004,"15min":4.2060000000000004},"last":1.337}]},{"osd":6,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.381,"15min":1.401},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8220000000000001,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":1.399},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5580000000000001,"15min":1.577},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.3029999999999999,"5min":4.335,"15min":4.335},"last":1.357}]},{"osd":7,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":3.7959999999999998,"5min":2.0779999999999998,"15min":1.792},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":40.311999999999998,"5min":40.311999999999998,"15min":40.311999999999998},"last":1.292},{"interface":"front","average":{"1min":3.7799999999999998,"5min":2.0840000000000001,"15min":1.802},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":40.286000000000001,"5min":40.286000000000001,"15min":40.286000000000001},"last":1.4610000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.83799999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.505}]},{"osd":1,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.288,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.99299999999999999,"15min":1.014},"min":{"1min":0.501,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4530000000000001,"5min":1.629,"15min":1.629},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.107,"15min":1.143},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.401,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2210000000000001,"15min":1.2250000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.1739999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5960000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.109},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2829999999999999,"15min":1.288},"min":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.8500000000000001,"5min":1.885,"15min":1.885},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":0.68200000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7849999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.222,"5min":1.349,"15min":1.3700000000000001},"min":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9550000000000001,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.1599999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.421,"15min":1.4410000000000001},"min":{"1min":0.76800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.224},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9079999999999999,"15min":1.613},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":40.671999999999997,"5min":40.671999999999997,"15min":40.671999999999997},"last":1.052}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.075},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91600000000000004,"15min":0.93600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.5469999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.94799999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.024},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3169999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.044,"15min":1.0780000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.569,"5min":2.278,"15min":2.278},"last":1.2709999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.353,"5min":1.847,"15min":1.847},"last":1.353},{"interface":"front","average":{"1min":1.032,"5min":1.232,"15min":1.2649999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7130000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.4079999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.323,"15min":1.345},"min":{"1min":0.69099999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7450000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.669},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.2090000000000001,"15min":1.2350000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.712,"5min":3.3839999999999999,"15min":3.3839999999999999},"last":1.712}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.345,"15min":1.361},"min":{"1min":0.87,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.821,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.536},{"interface":"front","average":{"1min":1.153,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8480000000000001,"5min":3.706,"15min":3.706},"last":1.452}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2869999999999999,"15min":1.319},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7729999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.613},{"interface":"front","average":{"1min":1.288,"5min":1.4019999999999999,"15min":1.421},"min":{"1min":0.872,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.913,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4870000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.46,"15min":1.5029999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.99,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.4119999999999999,"15min":1.4590000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.9910000000000001,"5min":3.661,"15min":3.661},"last":1.504}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96499999999999997,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3160000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.329},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.016,"15min":1.026},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.3120000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94499999999999995,"15min":0.96199999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.034,"15min":1.052},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.534,"5min":1.998,"15min":1.998},"last":1.4430000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.1140000000000001,"15min":1.137},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7589999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.079,"15min":1.087},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6870000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.226,"15min":1.26},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4650000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3720000000000001,"15min":1.3759999999999999},"min":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.905,"5min":2.415,"15min":2.415},"last":1.7450000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.365,"15min":1.3979999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.8360000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.496}]},{"osd":6,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.47,"15min":1.48},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1259999999999999,"5min":2.544,"15min":2.544},"last":1.395},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.4630000000000001,"15min":1.4890000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.909,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.575}]},{"osd":7,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.488,"15min":1.504},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0270000000000001,"5min":2.355,"15min":2.355},"last":2.0990000000000002},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5329999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.056,"5min":2.492,"15min":2.492},"last":1.5529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.94699999999999995,"15min":0.98699999999999999},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.133,"5min":2.54,"15min":2.54},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.95199999999999996,"15min":0.98799999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.599,"5min":1.607,"15min":1.607},"last":0.755}]},{"osd":2,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.0840000000000001,"15min":1.1319999999999999},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.2609999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.91200000000000003,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.107,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.375}]},{"osd":3,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.123},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4410000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.173,"15min":1.206},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5700000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.345}]},{"osd":4,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3660000000000001,"15min":1.403},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5209999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.494,"15min":1.542},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5209999999999999,"5min":2.456,"15min":2.456},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3999999999999999,"15min":1.427},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.125},{"interface":"front","average":{"1min":1.028,"5min":1.4390000000000001,"15min":1.5069999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.522}]},{"osd":6,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.587,"15min":1.665},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7470000000000001,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.5429999999999999,"15min":1.591},"min":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.8879999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.2869999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.4510000000000001,"15min":1.5089999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7929999999999999,"5min":2.802,"15min":2.802},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.631,"15min":1.698},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.8859999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.4930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:45.428 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:29:45.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:45.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.650+0000 7f4d6ffff640 1 Processor -- start 2023-11-24T06:29:45.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.651+0000 7f4d6ffff640 1 -- start start 2023-11-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d6ffff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d70143c40 0x7f4d70071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d6ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d700717d0 0x7f4d700cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d6ffff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d70070e50 con 0x7f4d700cca40 2023-11-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d6ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700723c0 con 0x7f4d700717d0 2023-11-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d6ffff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700cce40 con 0x7f4d70143c40 2023-11-24T06:29:45.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d67fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d700717d0 0x7f4d700cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d67fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d700717d0 0x7f4d700cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45936/0 (socket says 172.21.15.64:45936) 2023-11-24T06:29:45.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.653+0000 7f4d67fff640 1 -- 172.21.15.64:0/304665854 learned_addr learned my addr 172.21.15.64:0/304665854 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.654+0000 7f4d6effd640 1 --2- 172.21.15.64:0/304665854 >> v2:172.21.15.162:3300/0 conn(0x7f4d70143c40 0x7f4d70071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.654+0000 7f4d67fff640 1 -- 172.21.15.64:0/304665854 >> v2:172.21.15.162:3300/0 conn(0x7f4d70143c40 msgr2=0x7f4d70071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.654+0000 7f4d67fff640 1 --2- 172.21.15.64:0/304665854 >> v2:172.21.15.162:3300/0 conn(0x7f4d70143c40 0x7f4d70071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.654+0000 7f4d67fff640 1 -- 172.21.15.64:0/304665854 >> v1:172.21.15.162:6789/0 conn(0x7f4d700cca40 legacy=0x7f4d7015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.654+0000 7f4d67fff640 1 -- 172.21.15.64:0/304665854 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7006e170 con 0x7f4d700717d0 2023-11-24T06:29:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.655+0000 7f4d67fff640 1 --2- 172.21.15.64:0/304665854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d700cc500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4d7006e410 tx=0x7f4d5802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ab856ab6dbbbb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.655+0000 7f4d6e7fc640 1 -- 172.21.15.64:0/304665854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d580317e0 con 0x7f4d700717d0 2023-11-24T06:29:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.655+0000 7f4d6e7fc640 1 -- 172.21.15.64:0/304665854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58031940 con 0x7f4d700717d0 2023-11-24T06:29:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.655+0000 7f4d6e7fc640 1 -- 172.21.15.64:0/304665854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58031cd0 con 0x7f4d700717d0 2023-11-24T06:29:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 -- 172.21.15.64:0/304665854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 msgr2=0x7f4d700cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/304665854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d700cc500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4d7006e410 tx=0x7f4d5802b880 comp rx=0 tx=0).stop 2023-11-24T06:29:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 -- 172.21.15.64:0/304665854 shutdown_connections 2023-11-24T06:29:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/304665854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d700cc500 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/304665854 >> v2:172.21.15.162:3300/0 conn(0x7f4d70143c40 0x7f4d70071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 -- 172.21.15.64:0/304665854 >> 172.21.15.64:0/304665854 conn(0x7f4d7006a800 msgr2=0x7f4d7006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 -- 172.21.15.64:0/304665854 shutdown_connections 2023-11-24T06:29:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.656+0000 7f4d6ffff640 1 -- 172.21.15.64:0/304665854 wait complete. 2023-11-24T06:29:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.658+0000 7f4d6ffff640 1 Processor -- start 2023-11-24T06:29:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.658+0000 7f4d6ffff640 1 -- start start 2023-11-24T06:29:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d701e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d700cca40 0x7f4d701585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d70143c40 0x7f4d70141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7013b0a0 con 0x7f4d700717d0 2023-11-24T06:29:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6ffff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7013adc0 con 0x7f4d70143c40 2023-11-24T06:29:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6ffff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d701e45f0 con 0x7f4d700cca40 2023-11-24T06:29:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d701e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d701e50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45946/0 (socket says 172.21.15.64:45946) 2023-11-24T06:29:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.659+0000 7f4d6effd640 1 -- 172.21.15.64:0/2185901346 learned_addr learned my addr 172.21.15.64:0/2185901346 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6effd640 1 -- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d700cca40 msgr2=0x7f4d701585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6effd640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d700cca40 0x7f4d701585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6f7fe640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d70143c40 0x7f4d70141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6effd640 1 -- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d70143c40 msgr2=0x7f4d70141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6effd640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d70143c40 0x7f4d70141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6effd640 1 -- 172.21.15.64:0/2185901346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d580094d0 con 0x7f4d700717d0 2023-11-24T06:29:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6f7fe640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d70143c40 0x7f4d70141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d6effd640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d701e50d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000d5f0 tx=0x7f4d6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.660+0000 7f4d677fe640 1 -- 172.21.15.64:0/2185901346 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600125f0 con 0x7f4d700717d0 2023-11-24T06:29:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.661+0000 7f4d677fe640 1 -- 172.21.15.64:0/2185901346 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60012750 con 0x7f4d700717d0 2023-11-24T06:29:45.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.661+0000 7f4d677fe640 1 -- 172.21.15.64:0/2185901346 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60012b00 con 0x7f4d700717d0 2023-11-24T06:29:45.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.661+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d701e48d0 con 0x7f4d700717d0 2023-11-24T06:29:45.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.661+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d70158ae0 con 0x7f4d700717d0 2023-11-24T06:29:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.662+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d701e4450 con 0x7f4d700717d0 2023-11-24T06:29:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.663+0000 7f4d677fe640 1 -- 172.21.15.64:0/2185901346 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d60012c60 con 0x7f4d700717d0 2023-11-24T06:29:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.664+0000 7f4d677fe640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d5006ae40 0x7f4d5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.664+0000 7f4d677fe640 1 -- 172.21.15.64:0/2185901346 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4d60089800 con 0x7f4d700717d0 2023-11-24T06:29:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.664+0000 7f4d67fff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d5006ae40 0x7f4d5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.665+0000 7f4d67fff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d5006ae40 0x7f4d5006d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d58036000 tx=0x7f4d58030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.668+0000 7f4d677fe640 1 -- 172.21.15.64:0/2185901346 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d701e4450 con 0x7f4d700717d0 2023-11-24T06:29:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.858+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d700720e0 con 0x7f4d5006ae40 2023-11-24T06:29:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.864+0000 7f4d677fe640 1 -- 172.21.15.64:0/2185901346 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48335 (secure 0 0 0) 0x7f4d58031ab0 con 0x7f4d5006ae40 2023-11-24T06:29:45.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:45.868 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:29:45.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.866+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d5006ae40 msgr2=0x7f4d5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.866+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d5006ae40 0x7f4d5006d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d58036000 tx=0x7f4d58030000 comp rx=0 tx=0).stop 2023-11-24T06:29:45.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.866+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 msgr2=0x7f4d701e50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d701e50d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000d5f0 tx=0x7f4d6000dab0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 shutdown_connections 2023-11-24T06:29:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f4d5006ae40 0x7f4d5006d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d70143c40 0x7f4d70141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d700cca40 0x7f4d701585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 --2- 172.21.15.64:0/2185901346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d700717d0 0x7f4d701e50d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 >> 172.21.15.64:0/2185901346 conn(0x7f4d7006a800 msgr2=0x7f4d70141230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 shutdown_connections 2023-11-24T06:29:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:45.867+0000 7f4d6ffff640 1 -- 172.21.15.64:0/2185901346 wait complete. 2023-11-24T06:29:45.890 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-11-24T06:29:45.138242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220856,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753864,"statfs":{"total":773094113280,"available":772867956736,"internally_reserved":0,"allocated":4792320,"data_stored":2599216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4388,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002865"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-24T06:29:39.128458+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:29:39.128458+0000","last_peered":"2023-11-24T06:29:39.128458+0000","last_clean":"2023-11-24T06:29:39.128458+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:29:39.128458+0000","last_undegraded":"2023-11-24T06:29:39.128458+0000","last_fullsized":"2023-11-24T06:29:39.128458+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97399999999999998,"15min":0.97699999999999998},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.633,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0469999999999999,"15min":1.0589999999999999},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7669999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.292}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0529999999999999,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.3839999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.141,"15min":1.169},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.581,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.3680000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0960000000000001,"15min":1.091},"min":{"1min":0.64100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.1779999999999999,"15min":1.1859999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.995}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.145,"15min":1.149},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.347,"15min":1.369},"min":{"1min":0.65300000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.871,"5min":2.024,"15min":2.024},"last":1.139}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.758,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3380000000000001,"15min":1.343},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.379,"15min":1.383},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.093,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.254},{"interface":"front","average":{"1min":1.373,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.56599999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.853,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.4650000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.476,"15min":1.468},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.397,"15min":1.393},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.506}]}]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.526,"5min":2.323,"15min":2.323},"last":1.73},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97299999999999998,"15min":0.96999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2889999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.0700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5669999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":1.597}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.048,"15min":1.0329999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.131,"15min":1.1259999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.3160000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.151,"15min":1.167},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2290000000000001,"15min":1.2470000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.2410000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.288,"15min":1.302},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.379,"15min":1.3919999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.847,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.95399999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.425,"15min":1.446},"min":{"1min":0.77900000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.887,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.417,"15min":1.4370000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8440000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5289999999999999,"15min":1.5549999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.839,"5min":3.052,"15min":3.052},"last":1.347},{"interface":"front","average":{"1min":1.254,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.732,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.111}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.53100000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.413},{"interface":"front","average":{"1min":1.016,"5min":1.1120000000000001,"15min":1.1279999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.671,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.361}]},{"osd":1,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.105,"15min":1.1140000000000001},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.6439999999999999,"5min":1.79,"15min":1.79},"last":0.871},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.224,"15min":1.24},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8320000000000001,"5min":1.857,"15min":1.857},"last":1.5029999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.222,"15min":1.24},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.897,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.2589999999999999,"15min":1.2849999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6499999999999999,"5min":1.986,"15min":1.986},"last":1.1759999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.849,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.329,"15min":1.361},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.746,"5min":2.359,"15min":2.359},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.3260000000000001,"15min":1.3580000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.702,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.377,"15min":1.385},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1160000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.6459999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.4299999999999999,"15min":1.46},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9630000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.464,"15min":1.488},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9610000000000001,"5min":3.0299999999999998,"15min":3.0299999999999998},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5249999999999999,"15min":1.5389999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2570000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.5880000000000001,"15min":1.6220000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1970000000000001,"5min":2.552,"15min":2.552},"last":1.6140000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.113,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2210000000000001,"15min":1.2370000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64600000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.8879999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1459999999999999,"15min":1.1479999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.978,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.131}]},{"osd":2,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6870000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.244},"min":{"1min":0.69999999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0840000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.371},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2480000000000002,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.2130000000000001,"15min":1.23},"min":{"1min":0.58399999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0659999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.304}]},{"osd":5,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.454,"15min":1.474},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2400000000000002,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.51,"15min":1.53},"min":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.157,"5min":4.2060000000000004,"15min":4.2060000000000004},"last":1.337}]},{"osd":6,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.381,"15min":1.401},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8220000000000001,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":1.399},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5580000000000001,"15min":1.577},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.3029999999999999,"5min":4.335,"15min":4.335},"last":1.357}]},{"osd":7,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":3.7959999999999998,"5min":2.0779999999999998,"15min":1.792},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":40.311999999999998,"5min":40.311999999999998,"15min":40.311999999999998},"last":1.292},{"interface":"front","average":{"1min":3.7799999999999998,"5min":2.0840000000000001,"15min":1.802},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":40.286000000000001,"5min":40.286000000000001,"15min":40.286000000000001},"last":1.4610000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.83799999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.505}]},{"osd":1,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.288,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.99299999999999999,"15min":1.014},"min":{"1min":0.501,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4530000000000001,"5min":1.629,"15min":1.629},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.107,"15min":1.143},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.401,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2210000000000001,"15min":1.2250000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.1739999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5960000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.109},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2829999999999999,"15min":1.288},"min":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.8500000000000001,"5min":1.885,"15min":1.885},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":0.68200000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7849999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.222,"5min":1.349,"15min":1.3700000000000001},"min":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9550000000000001,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.1599999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.421,"15min":1.4410000000000001},"min":{"1min":0.76800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.224},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9079999999999999,"15min":1.613},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":40.671999999999997,"5min":40.671999999999997,"15min":40.671999999999997},"last":1.052}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.075},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91600000000000004,"15min":0.93600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.5469999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.94799999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.024},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3169999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.044,"15min":1.0780000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.569,"5min":2.278,"15min":2.278},"last":1.2709999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.353,"5min":1.847,"15min":1.847},"last":1.353},{"interface":"front","average":{"1min":1.032,"5min":1.232,"15min":1.2649999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7130000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.4079999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.323,"15min":1.345},"min":{"1min":0.69099999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7450000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.669},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.2090000000000001,"15min":1.2350000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.712,"5min":3.3839999999999999,"15min":3.3839999999999999},"last":1.712}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.345,"15min":1.361},"min":{"1min":0.87,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.821,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.536},{"interface":"front","average":{"1min":1.153,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8480000000000001,"5min":3.706,"15min":3.706},"last":1.452}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2869999999999999,"15min":1.319},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7729999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.613},{"interface":"front","average":{"1min":1.288,"5min":1.4019999999999999,"15min":1.421},"min":{"1min":0.872,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.913,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4870000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.46,"15min":1.5029999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.99,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.4119999999999999,"15min":1.4590000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.9910000000000001,"5min":3.661,"15min":3.661},"last":1.504}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96499999999999997,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3160000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.329},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.016,"15min":1.026},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.3120000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94499999999999995,"15min":0.96199999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.034,"15min":1.052},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.534,"5min":1.998,"15min":1.998},"last":1.4430000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.1140000000000001,"15min":1.137},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7589999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.079,"15min":1.087},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6870000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.226,"15min":1.26},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4650000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3720000000000001,"15min":1.3759999999999999},"min":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.905,"5min":2.415,"15min":2.415},"last":1.7450000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.365,"15min":1.3979999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.8360000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.496}]},{"osd":6,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.47,"15min":1.48},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1259999999999999,"5min":2.544,"15min":2.544},"last":1.395},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.4630000000000001,"15min":1.4890000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.909,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.575}]},{"osd":7,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.488,"15min":1.504},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0270000000000001,"5min":2.355,"15min":2.355},"last":2.0990000000000002},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5329999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.056,"5min":2.492,"15min":2.492},"last":1.5529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.94699999999999995,"15min":0.98699999999999999},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.133,"5min":2.54,"15min":2.54},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.95199999999999996,"15min":0.98799999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.599,"5min":1.607,"15min":1.607},"last":0.755}]},{"osd":2,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.0840000000000001,"15min":1.1319999999999999},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.2609999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.91200000000000003,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.107,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.375}]},{"osd":3,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.123},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4410000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.173,"15min":1.206},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5700000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.345}]},{"osd":4,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3660000000000001,"15min":1.403},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5209999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.494,"15min":1.542},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5209999999999999,"5min":2.456,"15min":2.456},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3999999999999999,"15min":1.427},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.125},{"interface":"front","average":{"1min":1.028,"5min":1.4390000000000001,"15min":1.5069999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.522}]},{"osd":6,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.587,"15min":1.665},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7470000000000001,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.5429999999999999,"15min":1.591},"min":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.8879999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.2869999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.4510000000000001,"15min":1.5089999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7929999999999999,"5min":2.802,"15min":2.802},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.631,"15min":1.698},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.8859999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.4930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:45.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:46.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.102+0000 7fa68759e640 1 Processor -- start 2023-11-24T06:29:46.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.102+0000 7fa68759e640 1 -- start start 2023-11-24T06:29:46.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.104+0000 7fa68759e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa688071410 0x7fa688146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.104+0000 7fa68759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa68806c7c0 0x7fa6880717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.104+0000 7fa68759e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68806cc50 con 0x7fa688071d30 2023-11-24T06:29:46.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.104+0000 7fa68759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68806cec0 con 0x7fa68806c7c0 2023-11-24T06:29:46.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.104+0000 7fa68759e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68806d1a0 con 0x7fa688071410 2023-11-24T06:29:46.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.104+0000 7fa67ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa68806c7c0 0x7fa6880717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa67ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa68806c7c0 0x7fa6880717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45954/0 (socket says 172.21.15.64:45954) 2023-11-24T06:29:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa67ffff640 1 -- 172.21.15.64:0/1994386930 learned_addr learned my addr 172.21.15.64:0/1994386930 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa68659c640 1 --2- 172.21.15.64:0/1994386930 >> v2:172.21.15.162:3300/0 conn(0x7fa688071410 0x7fa688146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa67ffff640 1 -- 172.21.15.64:0/1994386930 >> v2:172.21.15.162:3300/0 conn(0x7fa688071410 msgr2=0x7fa688146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa67ffff640 1 --2- 172.21.15.64:0/1994386930 >> v2:172.21.15.162:3300/0 conn(0x7fa688071410 0x7fa688146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa67ffff640 1 -- 172.21.15.64:0/1994386930 >> v1:172.21.15.162:6789/0 conn(0x7fa688071d30 legacy=0x7fa6880b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa67ffff640 1 -- 172.21.15.64:0/1994386930 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68806d480 con 0x7fa68806c7c0 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.105+0000 7fa68659c640 1 --2- 172.21.15.64:0/1994386930 >> v2:172.21.15.162:3300/0 conn(0x7fa688071410 0x7fa688146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.106+0000 7fa67ffff640 1 --2- 172.21.15.64:0/1994386930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68806c7c0 0x7fa6880717f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad5f0 tx=0x7fa67002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e284b205621fb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.106+0000 7fa685d9b640 1 -- 172.21.15.64:0/1994386930 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670031870 con 0x7fa68806c7c0 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa685d9b640 1 -- 172.21.15.64:0/1994386930 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6700319d0 con 0x7fa68806c7c0 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa685d9b640 1 -- 172.21.15.64:0/1994386930 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670031d60 con 0x7fa68806c7c0 2023-11-24T06:29:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 -- 172.21.15.64:0/1994386930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68806c7c0 msgr2=0x7fa6880717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 --2- 172.21.15.64:0/1994386930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68806c7c0 0x7fa6880717f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad5f0 tx=0x7fa67002b910 comp rx=0 tx=0).stop 2023-11-24T06:29:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 -- 172.21.15.64:0/1994386930 shutdown_connections 2023-11-24T06:29:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 --2- 172.21.15.64:0/1994386930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68806c7c0 0x7fa6880717f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 --2- 172.21.15.64:0/1994386930 >> v2:172.21.15.162:3300/0 conn(0x7fa688071410 0x7fa688146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 -- 172.21.15.64:0/1994386930 >> 172.21.15.64:0/1994386930 conn(0x7fa68806a800 msgr2=0x7fa68806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 -- 172.21.15.64:0/1994386930 shutdown_connections 2023-11-24T06:29:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.107+0000 7fa68759e640 1 -- 172.21.15.64:0/1994386930 wait complete. 2023-11-24T06:29:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.109+0000 7fa68759e640 1 Processor -- start 2023-11-24T06:29:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.109+0000 7fa68759e640 1 -- start start 2023-11-24T06:29:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.110+0000 7fa68759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa68806c7c0 0x7fa6881e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.110+0000 7fa68759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa688071410 0x7fa6880b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.110+0000 7fa68759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa688071d30 0x7fa68815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.110+0000 7fa68759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881e3360 con 0x7fa688071410 2023-11-24T06:29:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.110+0000 7fa68759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881e8410 con 0x7fa688071d30 2023-11-24T06:29:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.110+0000 7fa68759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881e86f0 con 0x7fa68806c7c0 2023-11-24T06:29:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa688071410 0x7fa6880b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa688071410 0x7fa6880b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45962/0 (socket says 172.21.15.64:45962) 2023-11-24T06:29:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 -- 172.21.15.64:0/1783343209 learned_addr learned my addr 172.21.15.64:0/1783343209 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa68659c640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa68806c7c0 0x7fa6881e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa686d9d640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa688071d30 0x7fa68815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 -- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa68806c7c0 msgr2=0x7fa6881e2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa68806c7c0 0x7fa6881e2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 -- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa688071d30 msgr2=0x7fa68815c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa688071d30 0x7fa68815c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 -- 172.21.15.64:0/1783343209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700094d0 con 0x7fa688071410 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa686d9d640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa688071d30 0x7fa68815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.111+0000 7fa67ffff640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa688071410 0x7fa6880b0030 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c00 tx=0x7fa6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.112+0000 7fa67effd640 1 -- 172.21.15.64:0/1783343209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670031840 con 0x7fa688071410 2023-11-24T06:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.112+0000 7fa67effd640 1 -- 172.21.15.64:0/1783343209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6700443f0 con 0x7fa688071410 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.112+0000 7fa67effd640 1 -- 172.21.15.64:0/1783343209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700447a0 con 0x7fa688071410 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.112+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6881e89d0 con 0x7fa688071410 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.112+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6881e8f90 con 0x7fa688071410 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.113+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6881e9180 con 0x7fa688071410 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.114+0000 7fa67effd640 1 -- 172.21.15.64:0/1783343209 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa67004d450 con 0x7fa688071410 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.115+0000 7fa67effd640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa66806ae40 0x7fa66806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.115+0000 7fa67effd640 1 -- 172.21.15.64:0/1783343209 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa67002d040 con 0x7fa688071410 2023-11-24T06:29:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.115+0000 7fa68659c640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa66806ae40 0x7fa66806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.116+0000 7fa68659c640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa66806ae40 0x7fa66806d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa6880b42d0 tx=0x7fa678008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.119+0000 7fa67effd640 1 -- 172.21.15.64:0/1783343209 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa670076e00 con 0x7fa688071410 2023-11-24T06:29:46.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.317+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa688067450 con 0x7fa688071410 2023-11-24T06:29:46.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.318+0000 7fa67effd640 1 -- 172.21.15.64:0/1783343209 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12962 (secure 0 0 0) 0x7fa670035020 con 0x7fa688071410 2023-11-24T06:29:46.321 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:46.321 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"fsid":"fb91be48-aa54-48b4-b765-8060df323172","created":"2023-11-24T06:27:22.005822+0000","modified":"2023-11-24T06:29:38.962984+0000","last_up_change":"2023-11-24T06:27:33.849089+0000","last_in_change":"2023-11-24T06:27:25.558077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:27:35.179412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21fc9639-fe05-492e-8fcc-f24a92c35d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6825","nonce":948599598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6827","nonce":948599598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6831","nonce":948599598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":948599598},{"type":"v1","addr":"172.21.15.64:6829","nonce":948599598}]},"public_addr":"172.21.15.64:6825/948599598","cluster_addr":"172.21.15.64:6827/948599598","heartbeat_back_addr":"172.21.15.64:6831/948599598","heartbeat_front_addr":"172.21.15.64:6829/948599598","state":["exists","up"]},{"osd":1,"uuid":"9ce466f0-b63b-4e7f-8dbc-50688e7358e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6809","nonce":1463145862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6811","nonce":1463145862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6815","nonce":1463145862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1463145862},{"type":"v1","addr":"172.21.15.64:6813","nonce":1463145862}]},"public_addr":"172.21.15.64:6809/1463145862","cluster_addr":"172.21.15.64:6811/1463145862","heartbeat_back_addr":"172.21.15.64:6815/1463145862","heartbeat_front_addr":"172.21.15.64:6813/1463145862","state":["exists","up"]},{"osd":2,"uuid":"30710438-2317-403e-bc18-09f2465b7000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6817","nonce":1175692500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6819","nonce":1175692500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6823","nonce":1175692500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1175692500},{"type":"v1","addr":"172.21.15.64:6821","nonce":1175692500}]},"public_addr":"172.21.15.64:6817/1175692500","cluster_addr":"172.21.15.64:6819/1175692500","heartbeat_back_addr":"172.21.15.64:6823/1175692500","heartbeat_front_addr":"172.21.15.64:6821/1175692500","state":["exists","up"]},{"osd":3,"uuid":"05951407-33c9-404c-8593-d959d76b5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6801","nonce":3744810476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6803","nonce":3744810476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6807","nonce":3744810476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3744810476},{"type":"v1","addr":"172.21.15.64:6805","nonce":3744810476}]},"public_addr":"172.21.15.64:6801/3744810476","cluster_addr":"172.21.15.64:6803/3744810476","heartbeat_back_addr":"172.21.15.64:6807/3744810476","heartbeat_front_addr":"172.21.15.64:6805/3744810476","state":["exists","up"]},{"osd":4,"uuid":"ea4ade87-f98b-48fc-8def-4bc9eaeef94a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6804","nonce":399042930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6810","nonce":399042930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6823","nonce":399042930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":399042930},{"type":"v1","addr":"172.21.15.162:6817","nonce":399042930}]},"public_addr":"172.21.15.162:6804/399042930","cluster_addr":"172.21.15.162:6810/399042930","heartbeat_back_addr":"172.21.15.162:6823/399042930","heartbeat_front_addr":"172.21.15.162:6817/399042930","state":["exists","up"]},{"osd":5,"uuid":"c8fce8ed-29fa-43b1-8bd8-dd84d4e0b42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6809","nonce":694094717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6814","nonce":694094717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6821","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6822","nonce":694094717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":694094717},{"type":"v1","addr":"172.21.15.162:6819","nonce":694094717}]},"public_addr":"172.21.15.162:6809/694094717","cluster_addr":"172.21.15.162:6814/694094717","heartbeat_back_addr":"172.21.15.162:6822/694094717","heartbeat_front_addr":"172.21.15.162:6819/694094717","state":["exists","up"]},{"osd":6,"uuid":"f592a84b-9a5e-4fc5-bc48-20ebf96843d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6801","nonce":1044826159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6805","nonce":1044826159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6815","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6818","nonce":1044826159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":1044826159},{"type":"v1","addr":"172.21.15.162:6812","nonce":1044826159}]},"public_addr":"172.21.15.162:6801/1044826159","cluster_addr":"172.21.15.162:6805/1044826159","heartbeat_back_addr":"172.21.15.162:6818/1044826159","heartbeat_front_addr":"172.21.15.162:6812/1044826159","state":["exists","up"]},{"osd":7,"uuid":"fd4a964a-aa39-4a85-bb27-c3f06856b9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6825","nonce":4255674423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6827","nonce":4255674423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6831","nonce":4255674423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4255674423},{"type":"v1","addr":"172.21.15.162:6829","nonce":4255674423}]},"public_addr":"172.21.15.162:6825/4255674423","cluster_addr":"172.21.15.162:6827/4255674423","heartbeat_back_addr":"172.21.15.162:6831/4255674423","heartbeat_front_addr":"172.21.15.162:6829/4255674423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.002500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.165953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.989421+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.108026+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.924591+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.992043+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:31.008502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:27:30.981655+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6836/2541249514":"2023-11-25T06:29:35.901267+0000","172.21.15.64:6834/2541249514":"2023-11-25T06:29:35.901267+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa66806ae40 msgr2=0x7fa66806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa66806ae40 0x7fa66806d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa6880b42d0 tx=0x7fa678008040 comp rx=0 tx=0).stop 2023-11-24T06:29:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa688071410 msgr2=0x7fa6880b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa688071410 0x7fa6880b0030 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c00 tx=0x7fa6700046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 shutdown_connections 2023-11-24T06:29:46.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fa66806ae40 0x7fa66806d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa688071d30 0x7fa68815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa688071410 0x7fa6880b0030 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 --2- 172.21.15.64:0/1783343209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa68806c7c0 0x7fa6881e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 >> 172.21.15.64:0/1783343209 conn(0x7fa68806a800 msgr2=0x7fa6880ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:46.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 shutdown_connections 2023-11-24T06:29:46.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:46.321+0000 7fa68759e640 1 -- 172.21.15.64:0/1783343209 wait complete. 2023-11-24T06:29:47.343 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T06:29:47.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.580+0000 7f706b147640 1 Processor -- start 2023-11-24T06:29:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.581+0000 7f706b147640 1 -- start start 2023-11-24T06:29:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.582+0000 7f706b147640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7064146770 0x7f706406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.582+0000 7f706b147640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70640b2e10 con 0x7f7064071f00 2023-11-24T06:29:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.582+0000 7f706b147640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7064146bc0 con 0x7f7064071410 2023-11-24T06:29:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.582+0000 7f706b147640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706406cec0 con 0x7f7064146770 2023-11-24T06:29:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.583+0000 7f706a946640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7064146770 0x7f706406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.583+0000 7f706a946640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7064146770 0x7f706406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45974/0 (socket says 172.21.15.64:45974) 2023-11-24T06:29:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.583+0000 7f7069944640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7064071f00 0x7f7064164ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35860/0 (socket says 172.21.15.64:35860) 2023-11-24T06:29:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.583+0000 7f706a946640 1 -- 172.21.15.64:0/4196505984 learned_addr learned my addr 172.21.15.64:0/4196505984 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.583+0000 7f706a946640 1 -- 172.21.15.64:0/4196505984 >> v1:172.21.15.162:6789/0 conn(0x7f7064071410 legacy=0x7f70640717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.583+0000 7f706a946640 1 -- 172.21.15.64:0/4196505984 >> v1:172.21.15.64:6789/0 conn(0x7f7064071f00 legacy=0x7f7064164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.584+0000 7f706a946640 1 -- 172.21.15.64:0/4196505984 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706406d1a0 con 0x7f7064146770 2023-11-24T06:29:47.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.583+0000 7f7069143640 1 -- 172.21.15.64:0/4196505984 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026320871 0 0) 0x7f70640b2e10 con 0x7f7064071f00 2023-11-24T06:29:47.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.585+0000 7f706a946640 1 --2- 172.21.15.64:0/4196505984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f706406c7c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f705c009e50 tx=0x7f705c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6c227896e6a448 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:47.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.585+0000 7f7069143640 1 -- 172.21.15.64:0/4196505984 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c02cb10 con 0x7f7064146770 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.585+0000 7f7069143640 1 -- 172.21.15.64:0/4196505984 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c02cc70 con 0x7f7064146770 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.585+0000 7f7069143640 1 -- 172.21.15.64:0/4196505984 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0343f0 con 0x7f7064146770 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.585+0000 7f706b147640 1 -- 172.21.15.64:0/4196505984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 msgr2=0x7f706406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.585+0000 7f706b147640 1 --2- 172.21.15.64:0/4196505984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f706406c7c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f705c009e50 tx=0x7f705c02bf70 comp rx=0 tx=0).stop 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.586+0000 7f706b147640 1 -- 172.21.15.64:0/4196505984 shutdown_connections 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.586+0000 7f706b147640 1 --2- 172.21.15.64:0/4196505984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f706406c7c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.586+0000 7f706b147640 1 -- 172.21.15.64:0/4196505984 >> 172.21.15.64:0/4196505984 conn(0x7f706406a800 msgr2=0x7f706406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:47.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.586+0000 7f706b147640 1 -- 172.21.15.64:0/4196505984 shutdown_connections 2023-11-24T06:29:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.586+0000 7f706b147640 1 -- 172.21.15.64:0/4196505984 wait complete. 2023-11-24T06:29:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.587+0000 7f706b147640 1 Processor -- start 2023-11-24T06:29:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.588+0000 7f706b147640 1 -- start start 2023-11-24T06:29:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706b147640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064071410 0x7f706415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706b147640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7064071f00 0x7f7064160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706b147640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f70640c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706b147640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641600e0 con 0x7f7064146770 2023-11-24T06:29:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706b147640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70640c3f10 con 0x7f7064071410 2023-11-24T06:29:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706b147640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706415fb20 con 0x7f7064071f00 2023-11-24T06:29:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f70640c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f70640c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45988/0 (socket says 172.21.15.64:45988) 2023-11-24T06:29:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 -- 172.21.15.64:0/2242015457 learned_addr learned my addr 172.21.15.64:0/2242015457 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 -- 172.21.15.64:0/2242015457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7064071f00 msgr2=0x7f7064160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7064071f00 0x7f7064160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a145640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064071410 0x7f706415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 -- 172.21.15.64:0/2242015457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064071410 msgr2=0x7f706415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064071410 0x7f706415ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.589+0000 7f706a946640 1 -- 172.21.15.64:0/2242015457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c009b30 con 0x7f7064146770 2023-11-24T06:29:47.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.590+0000 7f706a946640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f70640c49b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f705c036040 tx=0x7f705c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:47.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.590+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c03e450 con 0x7f7064146770 2023-11-24T06:29:47.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.590+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c03e5d0 con 0x7f7064146770 2023-11-24T06:29:47.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.590+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c03e980 con 0x7f7064146770 2023-11-24T06:29:47.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.590+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706415fe00 con 0x7f7064146770 2023-11-24T06:29:47.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.590+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70640c4490 con 0x7f7064146770 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.591+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f706406da40 con 0x7f7064146770 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.592+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f705c03eae0 con 0x7f7064146770 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.593+0000 7f705affd640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f704806ae40 0x7f704806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.593+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f705c0ac270 con 0x7f7064146770 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.594+0000 7f706a145640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f704806ae40 0x7f704806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.594+0000 7f705affd640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7f7048070ca0 0x7f70480730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.594+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 --> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7048073770 con 0x7f7048070ca0 2023-11-24T06:29:47.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.594+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f705c0ae850 con 0x7f7064146770 2023-11-24T06:29:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.594+0000 7f7069944640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7f7048070ca0 0x7f70480730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.594+0000 7f706a145640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f704806ae40 0x7f704806d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7060000f60 tx=0x7f7060009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.595+0000 7f7069944640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7f7048070ca0 0x7f70480730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.595+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== osd.0 v2:172.21.15.64:6824/948599598 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7048073770 con 0x7f7048070ca0 2023-11-24T06:29:47.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.610+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 --> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f706406d760 con 0x7f7048070ca0 2023-11-24T06:29:47.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.611+0000 7f705affd640 1 -- 172.21.15.64:0/2242015457 <== osd.0 v2:172.21.15.64:6824/948599598 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f70540070e0 con 0x7f7048070ca0 2023-11-24T06:29:47.614 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:47.614 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:47.614 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.611+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7f7048070ca0 msgr2=0x7f70480730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.611+0000 7f706b147640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6824/948599598,v1:172.21.15.64:6825/948599598] conn(0x7f7048070ca0 0x7f70480730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.611+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f704806ae40 msgr2=0x7f704806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.611+0000 7f706b147640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f704806ae40 0x7f704806d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7060000f60 tx=0x7f7060009040 comp rx=0 tx=0).stop 2023-11-24T06:29:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.611+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 msgr2=0x7f70640c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.611+0000 7f706b147640 1 --2- 172.21.15.64:0/2242015457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7064146770 0x7f70640c49b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f705c036040 tx=0x7f705c004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.612+0000 7f706a946640 1 -- 172.21.15.64:0/2242015457 reap_dead start 2023-11-24T06:29:47.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.612+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 shutdown_connections 2023-11-24T06:29:47.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.612+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 >> 172.21.15.64:0/2242015457 conn(0x7f706406a800 msgr2=0x7f706406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:47.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.612+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 shutdown_connections 2023-11-24T06:29:47.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.612+0000 7f706b147640 1 -- 172.21.15.64:0/2242015457 wait complete. 2023-11-24T06:29:47.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T06:29:47.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.850+0000 7fc993fff640 1 Processor -- start 2023-11-24T06:29:47.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.850+0000 7fc993fff640 1 -- start start 2023-11-24T06:29:47.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.852+0000 7fc993fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc994146850 0x7fc9940715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.852+0000 7fc993fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc994071b10 0x7fc9940ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.852+0000 7fc993fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc9940700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.852+0000 7fc993fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc994071fa0 con 0x7fc994146850 2023-11-24T06:29:47.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.852+0000 7fc993fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940b3490 con 0x7fc994071b10 2023-11-24T06:29:47.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.852+0000 7fc993fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940b3770 con 0x7fc9940adb00 2023-11-24T06:29:47.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc994146850 0x7fc9940715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc994146850 0x7fc9940715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46002/0 (socket says 172.21.15.64:46002) 2023-11-24T06:29:47.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 -- 172.21.15.64:0/4224761619 learned_addr learned my addr 172.21.15.64:0/4224761619 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:47.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc9937fe640 1 --2- 172.21.15.64:0/4224761619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc9940700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc98bfff640 1 --2- 172.21.15.64:0/4224761619 >> v2:172.21.15.162:3300/0 conn(0x7fc994071b10 0x7fc9940ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 -- 172.21.15.64:0/4224761619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 msgr2=0x7fc9940700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:47.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 --2- 172.21.15.64:0/4224761619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc9940700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 -- 172.21.15.64:0/4224761619 >> v2:172.21.15.162:3300/0 conn(0x7fc994071b10 msgr2=0x7fc9940ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:47.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 --2- 172.21.15.64:0/4224761619 >> v2:172.21.15.162:3300/0 conn(0x7fc994071b10 0x7fc9940ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.853+0000 7fc992ffd640 1 -- 172.21.15.64:0/4224761619 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940b3a50 con 0x7fc994146850 2023-11-24T06:29:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.855+0000 7fc992ffd640 1 --2- 172.21.15.64:0/4224761619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994146850 0x7fc9940715d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc994160cc0 tx=0x7fc98402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bd5302cc35e356 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.855+0000 7fc9927fc640 1 -- 172.21.15.64:0/4224761619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984031870 con 0x7fc994146850 2023-11-24T06:29:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.855+0000 7fc9927fc640 1 -- 172.21.15.64:0/4224761619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9840319d0 con 0x7fc994146850 2023-11-24T06:29:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.855+0000 7fc9927fc640 1 -- 172.21.15.64:0/4224761619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984031d60 con 0x7fc994146850 2023-11-24T06:29:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 -- 172.21.15.64:0/4224761619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994146850 msgr2=0x7fc9940715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 --2- 172.21.15.64:0/4224761619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994146850 0x7fc9940715d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc994160cc0 tx=0x7fc98402b910 comp rx=0 tx=0).stop 2023-11-24T06:29:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 -- 172.21.15.64:0/4224761619 shutdown_connections 2023-11-24T06:29:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 --2- 172.21.15.64:0/4224761619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc9940700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 --2- 172.21.15.64:0/4224761619 >> v2:172.21.15.162:3300/0 conn(0x7fc994071b10 0x7fc9940ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 --2- 172.21.15.64:0/4224761619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994146850 0x7fc9940715d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 -- 172.21.15.64:0/4224761619 >> 172.21.15.64:0/4224761619 conn(0x7fc99406a3e0 msgr2=0x7fc99406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 -- 172.21.15.64:0/4224761619 shutdown_connections 2023-11-24T06:29:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.856+0000 7fc993fff640 1 -- 172.21.15.64:0/4224761619 wait complete. 2023-11-24T06:29:47.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.858+0000 7fc993fff640 1 Processor -- start 2023-11-24T06:29:47.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.858+0000 7fc993fff640 1 -- start start 2023-11-24T06:29:47.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.859+0000 7fc993fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994071b10 0x7fc9941569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.859+0000 7fc993fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc994140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.859+0000 7fc993fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc994146850 0x7fc9941406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.859+0000 7fc993fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99413e5f0 con 0x7fc994071b10 2023-11-24T06:29:47.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.859+0000 7fc993fff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994156ef0 con 0x7fc994146850 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.859+0000 7fc993fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99413d9e0 con 0x7fc9940adb00 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.859+0000 7fc992ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994071b10 0x7fc9941569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994071b10 0x7fc9941569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46018/0 (socket says 172.21.15.64:46018) 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 -- 172.21.15.64:0/452758710 learned_addr learned my addr 172.21.15.64:0/452758710 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc9937fe640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc994146850 0x7fc9941406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc98bfff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc994140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 -- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 msgr2=0x7fc994140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc994140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 -- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc994146850 msgr2=0x7fc9941406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc994146850 0x7fc9941406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 -- 172.21.15.64:0/452758710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840094d0 con 0x7fc994071b10 2023-11-24T06:29:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc9937fe640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc994146850 0x7fc9941406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.860+0000 7fc992ffd640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994071b10 0x7fc9941569b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc98402b950 tx=0x7fc9840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.861+0000 7fc98b7fe640 1 -- 172.21.15.64:0/452758710 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98403ad30 con 0x7fc994071b10 2023-11-24T06:29:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.861+0000 7fc98b7fe640 1 -- 172.21.15.64:0/452758710 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9840443f0 con 0x7fc994071b10 2023-11-24T06:29:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.861+0000 7fc98b7fe640 1 -- 172.21.15.64:0/452758710 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9840447a0 con 0x7fc994071b10 2023-11-24T06:29:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.861+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99413dcc0 con 0x7fc994071b10 2023-11-24T06:29:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.861+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99413e280 con 0x7fc994071b10 2023-11-24T06:29:47.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.862+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994067450 con 0x7fc994071b10 2023-11-24T06:29:47.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.863+0000 7fc98b7fe640 1 -- 172.21.15.64:0/452758710 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc98404d450 con 0x7fc994071b10 2023-11-24T06:29:47.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.864+0000 7fc98b7fe640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc97406ae40 0x7fc97406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:47.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.864+0000 7fc98b7fe640 1 -- 172.21.15.64:0/452758710 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc98402d040 con 0x7fc994071b10 2023-11-24T06:29:47.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.864+0000 7fc98bfff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc97406ae40 0x7fc97406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:47.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.865+0000 7fc98bfff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc97406ae40 0x7fc97406d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc97c006fd0 tx=0x7fc97c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:47.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:47.869+0000 7fc98b7fe640 1 -- 172.21.15.64:0/452758710 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc984076cd0 con 0x7fc994071b10 2023-11-24T06:29:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.064+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9940b4230 con 0x7fc97406ae40 2023-11-24T06:29:48.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.064+0000 7fc98b7fe640 1 -- 172.21.15.64:0/452758710 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc97c00f830 con 0x7fc97406ae40 2023-11-24T06:29:48.126 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T06:29:48.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc97406ae40 msgr2=0x7fc97406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc97406ae40 0x7fc97406d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc97c006fd0 tx=0x7fc97c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:48.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994071b10 msgr2=0x7fc9941569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994071b10 0x7fc9941569b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc98402b950 tx=0x7fc9840046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 shutdown_connections 2023-11-24T06:29:48.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc97406ae40 0x7fc97406d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc994146850 0x7fc9941406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9940adb00 0x7fc994140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 --2- 172.21.15.64:0/452758710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc994071b10 0x7fc9941569b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 >> 172.21.15.64:0/452758710 conn(0x7fc99406a3e0 msgr2=0x7fc9940af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 shutdown_connections 2023-11-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.067+0000 7fc993fff640 1 -- 172.21.15.64:0/452758710 wait complete. 2023-11-24T06:29:48.127 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T06:29:48.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.336+0000 7f6691490640 1 Processor -- start 2023-11-24T06:29:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.336+0000 7f6691490640 1 -- start start 2023-11-24T06:29:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f6691490640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f668c0cf9a0 0x7f668c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f6691490640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 0x7f668c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f6691490640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0b1ee0 con 0x7f668c143ae0 2023-11-24T06:29:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f6691490640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668c0b2780 con 0x7f668c0cf9a0 2023-11-24T06:29:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f6691490640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c15e990 con 0x7f668c15e1e0 2023-11-24T06:29:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f668affd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f668c143ae0 0x7f668c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35870/0 (socket says 172.21.15.64:35870) 2023-11-24T06:29:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f668affd640 1 -- 172.21.15.64:0/3147267017 learned_addr learned my addr 172.21.15.64:0/3147267017 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.338+0000 7f66827fc640 1 --2- 172.21.15.64:0/3147267017 >> v2:172.21.15.64:3300/0 conn(0x7f668c0cf9a0 0x7f668c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.339+0000 7f668b7fe640 1 --2- 172.21.15.64:0/3147267017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 0x7f668c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.339+0000 7f668a7fc640 1 -- 172.21.15.64:0/3147267017 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396053508 0 0) 0x7f668c0b1ee0 con 0x7f668c143ae0 2023-11-24T06:29:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.339+0000 7f668a7fc640 1 -- 172.21.15.64:0/3147267017 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6678005730 con 0x7f668c143ae0 2023-11-24T06:29:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.339+0000 7f66827fc640 1 -- 172.21.15.64:0/3147267017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 msgr2=0x7f668c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.339+0000 7f66827fc640 1 --2- 172.21.15.64:0/3147267017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 0x7f668c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.339+0000 7f66827fc640 1 -- 172.21.15.64:0/3147267017 >> v1:172.21.15.64:6789/0 conn(0x7f668c143ae0 legacy=0x7f668c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.339+0000 7f66827fc640 1 -- 172.21.15.64:0/3147267017 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0b2130 con 0x7f668c0cf9a0 2023-11-24T06:29:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.340+0000 7f66827fc640 1 --2- 172.21.15.64:0/3147267017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c15dca0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f66740097a0 tx=0x7f667402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990074b175daf84b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.340+0000 7f668a7fc640 1 -- 172.21.15.64:0/3147267017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674031830 con 0x7f668c0cf9a0 2023-11-24T06:29:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.340+0000 7f668a7fc640 1 -- 172.21.15.64:0/3147267017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674031990 con 0x7f668c0cf9a0 2023-11-24T06:29:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.340+0000 7f668a7fc640 1 -- 172.21.15.64:0/3147267017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674031d20 con 0x7f668c0cf9a0 2023-11-24T06:29:48.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 -- 172.21.15.64:0/3147267017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 msgr2=0x7f668c15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 --2- 172.21.15.64:0/3147267017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c15dca0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f66740097a0 tx=0x7f667402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 -- 172.21.15.64:0/3147267017 shutdown_connections 2023-11-24T06:29:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 --2- 172.21.15.64:0/3147267017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 0x7f668c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 --2- 172.21.15.64:0/3147267017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c15dca0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 -- 172.21.15.64:0/3147267017 >> 172.21.15.64:0/3147267017 conn(0x7f668c06a800 msgr2=0x7f668c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 -- 172.21.15.64:0/3147267017 shutdown_connections 2023-11-24T06:29:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.341+0000 7f6691490640 1 -- 172.21.15.64:0/3147267017 wait complete. 2023-11-24T06:29:48.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.343+0000 7f6691490640 1 Processor -- start 2023-11-24T06:29:48.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.343+0000 7f6691490640 1 -- start start 2023-11-24T06:29:48.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.344+0000 7f6691490640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.344+0000 7f6691490640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f668c143ae0 0x7f668c0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.344+0000 7f6691490640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 0x7f668c1f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.344+0000 7f6691490640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1f4140 con 0x7f668c0cf9a0 2023-11-24T06:29:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.344+0000 7f6691490640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1f3630 con 0x7f668c143ae0 2023-11-24T06:29:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.344+0000 7f6691490640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1f38a0 con 0x7f668c15e1e0 2023-11-24T06:29:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46042/0 (socket says 172.21.15.64:46042) 2023-11-24T06:29:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 -- 172.21.15.64:0/3216120862 learned_addr learned my addr 172.21.15.64:0/3216120862 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f66827fc640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f668c143ae0 0x7f668c0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668b7fe640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 0x7f668c1f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 -- 172.21.15.64:0/3216120862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 msgr2=0x7f668c1f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f668c15e1e0 0x7f668c1f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 -- 172.21.15.64:0/3216120862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f668c143ae0 msgr2=0x7f668c0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f668c143ae0 0x7f668c0ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 -- 172.21.15.64:0/3216120862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6674009400 con 0x7f668c0cf9a0 2023-11-24T06:29:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f66827fc640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f668c143ae0 0x7f668c0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.345+0000 7f668affd640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c0ad8e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f667c00cbc0 tx=0x7f667c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.346+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00ad40 con 0x7f668c0cf9a0 2023-11-24T06:29:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.346+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c012400 con 0x7f668c0cf9a0 2023-11-24T06:29:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.346+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00aea0 con 0x7f668c0cf9a0 2023-11-24T06:29:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.346+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c1f3b80 con 0x7f668c0cf9a0 2023-11-24T06:29:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.346+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c0cc7b0 con 0x7f668c0cf9a0 2023-11-24T06:29:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.347+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f668c0b1ab0 con 0x7f668c0cf9a0 2023-11-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.348+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f667c007070 con 0x7f668c0cf9a0 2023-11-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.349+0000 7f6683fff640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f666c06ae40 0x7f666c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.349+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f667c089300 con 0x7f668c0cf9a0 2023-11-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.349+0000 7f66827fc640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f666c06ae40 0x7f666c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.349+0000 7f6683fff640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f666c070ca0 0x7f666c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.349+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 --> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f666c073770 con 0x7f666c070ca0 2023-11-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.349+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f667c08b8e0 con 0x7f668c0cf9a0 2023-11-24T06:29:48.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.350+0000 7f668b7fe640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f666c070ca0 0x7f666c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.350+0000 7f66827fc640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f666c06ae40 0x7f666c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6674036000 tx=0x7f6674008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.350+0000 7f668b7fe640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f666c070ca0 0x7f666c0730c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.351+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== osd.1 v2:172.21.15.64:6808/1463145862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f666c073770 con 0x7f666c070ca0 2023-11-24T06:29:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.366+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 --> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f668c073c60 con 0x7f666c070ca0 2023-11-24T06:29:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.366+0000 7f6683fff640 1 -- 172.21.15.64:0/3216120862 <== osd.1 v2:172.21.15.64:6808/1463145862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f668c073c60 con 0x7f666c070ca0 2023-11-24T06:29:48.369 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:48.369 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:48.369 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.366+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f666c070ca0 msgr2=0x7f666c0730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.366+0000 7f6691490640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6808/1463145862,v1:172.21.15.64:6809/1463145862] conn(0x7f666c070ca0 0x7f666c0730c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f666c06ae40 msgr2=0x7f666c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f666c06ae40 0x7f666c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6674036000 tx=0x7f6674008030 comp rx=0 tx=0).stop 2023-11-24T06:29:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 msgr2=0x7f668c0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 --2- 172.21.15.64:0/3216120862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f668c0cf9a0 0x7f668c0ad8e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f667c00cbc0 tx=0x7f667c00a430 comp rx=0 tx=0).stop 2023-11-24T06:29:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f668b7fe640 1 -- 172.21.15.64:0/3216120862 reap_dead start 2023-11-24T06:29:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 shutdown_connections 2023-11-24T06:29:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 >> 172.21.15.64:0/3216120862 conn(0x7f668c06a800 msgr2=0x7f668c15b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 shutdown_connections 2023-11-24T06:29:48.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.367+0000 7f6691490640 1 -- 172.21.15.64:0/3216120862 wait complete. 2023-11-24T06:29:48.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T06:29:48.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.573+0000 7fccf1743640 1 Processor -- start 2023-11-24T06:29:48.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.574+0000 7fccf1743640 1 -- start start 2023-11-24T06:29:48.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.575+0000 7fccf1743640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fccec0d0b60 0x7fccec0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.575+0000 7fccf1743640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fccec0aede0 0x7fccec06f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.575+0000 7fccf1743640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec071de0 con 0x7fccec06a3e0 2023-11-24T06:29:48.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.575+0000 7fccf1743640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0af270 con 0x7fccec0aede0 2023-11-24T06:29:48.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.575+0000 7fccf1743640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec072030 con 0x7fccec0d0b60 2023-11-24T06:29:48.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fcceb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fccec0aede0 0x7fccec06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fcceb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fccec0aede0 0x7fccec06f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46058/0 (socket says 172.21.15.64:46058) 2023-11-24T06:29:48.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fcceb7fe640 1 -- 172.21.15.64:0/1112968044 learned_addr learned my addr 172.21.15.64:0/1112968044 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:48.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fccea7fc640 1 --2- 172.21.15.64:0/1112968044 >> v2:172.21.15.162:3300/0 conn(0x7fccec0d0b60 0x7fccec0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fcceb7fe640 1 -- 172.21.15.64:0/1112968044 >> v2:172.21.15.162:3300/0 conn(0x7fccec0d0b60 msgr2=0x7fccec0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:48.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fcceb7fe640 1 --2- 172.21.15.64:0/1112968044 >> v2:172.21.15.162:3300/0 conn(0x7fccec0d0b60 0x7fccec0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fcceb7fe640 1 -- 172.21.15.64:0/1112968044 >> v1:172.21.15.162:6789/0 conn(0x7fccec06a3e0 legacy=0x7fccec06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:48.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.576+0000 7fcceb7fe640 1 -- 172.21.15.64:0/1112968044 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec072310 con 0x7fccec0aede0 2023-11-24T06:29:48.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fcceb7fe640 1 --2- 172.21.15.64:0/1112968044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec0aede0 0x7fccec06f080 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fccec0b6120 tx=0x7fcce402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff7fead8503bf47 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fcce9ffb640 1 -- 172.21.15.64:0/1112968044 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4031870 con 0x7fccec0aede0 2023-11-24T06:29:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fcce9ffb640 1 -- 172.21.15.64:0/1112968044 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce40319d0 con 0x7fccec0aede0 2023-11-24T06:29:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fcce9ffb640 1 -- 172.21.15.64:0/1112968044 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4031d60 con 0x7fccec0aede0 2023-11-24T06:29:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fccf1743640 1 -- 172.21.15.64:0/1112968044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec0aede0 msgr2=0x7fccec06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fccf1743640 1 --2- 172.21.15.64:0/1112968044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec0aede0 0x7fccec06f080 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fccec0b6120 tx=0x7fcce402b910 comp rx=0 tx=0).stop 2023-11-24T06:29:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fccf1743640 1 -- 172.21.15.64:0/1112968044 shutdown_connections 2023-11-24T06:29:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fccf1743640 1 --2- 172.21.15.64:0/1112968044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec0aede0 0x7fccec06f080 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fccf1743640 1 --2- 172.21.15.64:0/1112968044 >> v2:172.21.15.162:3300/0 conn(0x7fccec0d0b60 0x7fccec0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.578+0000 7fccf1743640 1 -- 172.21.15.64:0/1112968044 >> 172.21.15.64:0/1112968044 conn(0x7fccec0cfef0 msgr2=0x7fccec0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.579+0000 7fccf1743640 1 -- 172.21.15.64:0/1112968044 shutdown_connections 2023-11-24T06:29:48.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.579+0000 7fccf1743640 1 -- 172.21.15.64:0/1112968044 wait complete. 2023-11-24T06:29:48.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.580+0000 7fccf1743640 1 Processor -- start 2023-11-24T06:29:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.580+0000 7fccf1743640 1 -- start start 2023-11-24T06:29:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.581+0000 7fccf1743640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec06a3e0 0x7fccec147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.581+0000 7fccf1743640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec0aede0 0x7fccec1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.581+0000 7fccf1743640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccec0d0b60 0x7fccec0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.581+0000 7fccf1743640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0cd950 con 0x7fccec06a3e0 2023-11-24T06:29:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.581+0000 7fccf1743640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec147ad0 con 0x7fccec0aede0 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.581+0000 7fccf1743640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0ccd40 con 0x7fccec0d0b60 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec06a3e0 0x7fccec147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec06a3e0 0x7fccec147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46070/0 (socket says 172.21.15.64:46070) 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 -- 172.21.15.64:0/4082197638 learned_addr learned my addr 172.21.15.64:0/4082197638 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fccea7fc640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec0aede0 0x7fccec1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceb7fe640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccec0d0b60 0x7fccec0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 -- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccec0d0b60 msgr2=0x7fccec0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccec0d0b60 0x7fccec0cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 -- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec0aede0 msgr2=0x7fccec1441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec0aede0 0x7fccec1441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 -- 172.21.15.64:0/4082197638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40094d0 con 0x7fccec06a3e0 2023-11-24T06:29:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.582+0000 7fcceaffd640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec06a3e0 0x7fccec147590 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00ba50 tx=0x7fccdc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.583+0000 7fcce37fe640 1 -- 172.21.15.64:0/4082197638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00cbe0 con 0x7fccec06a3e0 2023-11-24T06:29:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.583+0000 7fcce37fe640 1 -- 172.21.15.64:0/4082197638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc00cd80 con 0x7fccec06a3e0 2023-11-24T06:29:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.583+0000 7fcce37fe640 1 -- 172.21.15.64:0/4082197638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc014680 con 0x7fccec06a3e0 2023-11-24T06:29:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.583+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec0cd020 con 0x7fccec06a3e0 2023-11-24T06:29:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.583+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec0cd5e0 con 0x7fccec06a3e0 2023-11-24T06:29:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.584+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec0675d0 con 0x7fccec06a3e0 2023-11-24T06:29:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.585+0000 7fcce37fe640 1 -- 172.21.15.64:0/4082197638 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fccdc0147e0 con 0x7fccec06a3e0 2023-11-24T06:29:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.586+0000 7fcce37fe640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fcccc06ae40 0x7fcccc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.586+0000 7fcce37fe640 1 -- 172.21.15.64:0/4082197638 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fccdc0890f0 con 0x7fccec06a3e0 2023-11-24T06:29:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.586+0000 7fccea7fc640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fcccc06ae40 0x7fcccc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.587+0000 7fccea7fc640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fcccc06ae40 0x7fcccc06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fccec1f6350 tx=0x7fccd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.590+0000 7fcce37fe640 1 -- 172.21.15.64:0/4082197638 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fccdc0545e0 con 0x7fccec06a3e0 2023-11-24T06:29:48.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.782+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fccec06eb60 con 0x7fcccc06ae40 2023-11-24T06:29:48.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.783+0000 7fcce37fe640 1 -- 172.21.15.64:0/4082197638 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fccd4009d60 con 0x7fcccc06ae40 2023-11-24T06:29:48.786 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T06:29:48.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fcccc06ae40 msgr2=0x7fcccc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fcccc06ae40 0x7fcccc06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fccec1f6350 tx=0x7fccd4008040 comp rx=0 tx=0).stop 2023-11-24T06:29:48.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec06a3e0 msgr2=0x7fccec147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec06a3e0 0x7fccec147590 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00ba50 tx=0x7fccdc00bf10 comp rx=0 tx=0).stop 2023-11-24T06:29:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 shutdown_connections 2023-11-24T06:29:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fcccc06ae40 0x7fcccc06d300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccec0d0b60 0x7fccec0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec0aede0 0x7fccec1441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 --2- 172.21.15.64:0/4082197638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec06a3e0 0x7fccec147590 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.787+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 >> 172.21.15.64:0/4082197638 conn(0x7fccec0cfef0 msgr2=0x7fccec0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.788+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 shutdown_connections 2023-11-24T06:29:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:48.788+0000 7fccf1743640 1 -- 172.21.15.64:0/4082197638 wait complete. 2023-11-24T06:29:48.814 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T06:29:48.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:49.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.021+0000 7f9df5b53640 1 Processor -- start 2023-11-24T06:29:49.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.021+0000 7f9df5b53640 1 -- start start 2023-11-24T06:29:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.023+0000 7f9df5b53640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9df013df80 0x7f9df015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.023+0000 7f9df5b53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df00cf9a0 0x7f9df00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.023+0000 7f9df5b53640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df006c470 con 0x7f9df013e360 2023-11-24T06:29:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.023+0000 7f9df5b53640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df006c750 con 0x7f9df013df80 2023-11-24T06:29:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.023+0000 7f9df5b53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df006ca30 con 0x7f9df00cf9a0 2023-11-24T06:29:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.023+0000 7f9def7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9df013df80 0x7f9df015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.024+0000 7f9def7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9df013df80 0x7f9df015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46076/0 (socket says 172.21.15.64:46076) 2023-11-24T06:29:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.024+0000 7f9def7fe640 1 -- 172.21.15.64:0/1274607032 learned_addr learned my addr 172.21.15.64:0/1274607032 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.024+0000 7f9deffff640 1 --2- 172.21.15.64:0/1274607032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df00cf9a0 0x7f9df00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.024+0000 7f9def7fe640 1 -- 172.21.15.64:0/1274607032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df00cf9a0 msgr2=0x7f9df00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.024+0000 7f9def7fe640 1 --2- 172.21.15.64:0/1274607032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df00cf9a0 0x7f9df00cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.024+0000 7f9def7fe640 1 -- 172.21.15.64:0/1274607032 >> v1:172.21.15.162:6789/0 conn(0x7f9df013e360 legacy=0x7f9df00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.024+0000 7f9def7fe640 1 -- 172.21.15.64:0/1274607032 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df006cd10 con 0x7f9df013df80 2023-11-24T06:29:49.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.026+0000 7f9def7fe640 1 --2- 172.21.15.64:0/1274607032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013df80 0x7f9df015bee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4040f7ead1a5f1ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.027+0000 7f9deeffd640 1 -- 172.21.15.64:0/1274607032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0031640 con 0x7f9df013df80 2023-11-24T06:29:49.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.027+0000 7f9deeffd640 1 -- 172.21.15.64:0/1274607032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00317a0 con 0x7f9df013df80 2023-11-24T06:29:49.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.027+0000 7f9deeffd640 1 -- 172.21.15.64:0/1274607032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0031b30 con 0x7f9df013df80 2023-11-24T06:29:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 -- 172.21.15.64:0/1274607032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013df80 msgr2=0x7f9df015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 --2- 172.21.15.64:0/1274607032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013df80 0x7f9df015bee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 -- 172.21.15.64:0/1274607032 shutdown_connections 2023-11-24T06:29:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 --2- 172.21.15.64:0/1274607032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df00cf9a0 0x7f9df00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 --2- 172.21.15.64:0/1274607032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013df80 0x7f9df015bee0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 -- 172.21.15.64:0/1274607032 >> 172.21.15.64:0/1274607032 conn(0x7f9df006a800 msgr2=0x7f9df006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 -- 172.21.15.64:0/1274607032 shutdown_connections 2023-11-24T06:29:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.028+0000 7f9df5b53640 1 -- 172.21.15.64:0/1274607032 wait complete. 2023-11-24T06:29:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.030+0000 7f9df5b53640 1 Processor -- start 2023-11-24T06:29:49.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.030+0000 7f9df5b53640 1 -- start start 2023-11-24T06:29:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.031+0000 7f9df5b53640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9df00cf9a0 0x7f9df014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.031+0000 7f9df5b53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df013df80 0x7f9df0152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.031+0000 7f9df5b53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013e360 0x7f9df0130600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.031+0000 7f9df5b53640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0153850 con 0x7f9df013e360 2023-11-24T06:29:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.031+0000 7f9df5b53640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df01508e0 con 0x7f9df00cf9a0 2023-11-24T06:29:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.031+0000 7f9df5b53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0152c40 con 0x7f9df013df80 2023-11-24T06:29:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013e360 0x7f9df0130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013e360 0x7f9df0130600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46082/0 (socket says 172.21.15.64:46082) 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 -- 172.21.15.64:0/1283509373 learned_addr learned my addr 172.21.15.64:0/1283509373 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9de7fff640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df013df80 0x7f9df0152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9def7fe640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9df00cf9a0 0x7f9df014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 -- 172.21.15.64:0/1283509373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df013df80 msgr2=0x7f9df0152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9df013df80 0x7f9df0152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 -- 172.21.15.64:0/1283509373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9df00cf9a0 msgr2=0x7f9df014ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9df00cf9a0 0x7f9df014ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 -- 172.21.15.64:0/1283509373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00094d0 con 0x7f9df013e360 2023-11-24T06:29:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9def7fe640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9df00cf9a0 0x7f9df014ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.032+0000 7f9deffff640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013e360 0x7f9df0130600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9de8007d80 tx=0x7f9de800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.033+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8014420 con 0x7f9df013e360 2023-11-24T06:29:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.033+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8014580 con 0x7f9df013e360 2023-11-24T06:29:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.033+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8014930 con 0x7f9df013e360 2023-11-24T06:29:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.033+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df0152f20 con 0x7f9df013e360 2023-11-24T06:29:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.033+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df01534e0 con 0x7f9df013e360 2023-11-24T06:29:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.034+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9df015f9c0 con 0x7f9df013e360 2023-11-24T06:29:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.035+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9de8014a90 con 0x7f9df013e360 2023-11-24T06:29:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.036+0000 7f9decff9640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dd006ae70 0x7f9dd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.036+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9de8089c30 con 0x7f9df013e360 2023-11-24T06:29:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.036+0000 7f9def7fe640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dd006ae70 0x7f9dd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.036+0000 7f9decff9640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9dd0070cd0 0x7f9dd00730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.036+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 --> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9dd00737a0 con 0x7f9dd0070cd0 2023-11-24T06:29:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.036+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de808e2a0 con 0x7f9df013e360 2023-11-24T06:29:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.036+0000 7f9de7fff640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9dd0070cd0 0x7f9dd00730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.037+0000 7f9def7fe640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dd006ae70 0x7f9dd006d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9de0036000 tx=0x7f9de003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.037+0000 7f9de7fff640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9dd0070cd0 0x7f9dd00730f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.038+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== osd.2 v2:172.21.15.64:6816/1175692500 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9dd00737a0 con 0x7f9dd0070cd0 2023-11-24T06:29:49.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.052+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 --> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9df0153a00 con 0x7f9dd0070cd0 2023-11-24T06:29:49.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.053+0000 7f9decff9640 1 -- 172.21.15.64:0/1283509373 <== osd.2 v2:172.21.15.64:6816/1175692500 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9df0153a00 con 0x7f9dd0070cd0 2023-11-24T06:29:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:49.056 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.053+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9dd0070cd0 msgr2=0x7f9dd00730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6816/1175692500,v1:172.21.15.64:6817/1175692500] conn(0x7f9dd0070cd0 0x7f9dd00730f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dd006ae70 msgr2=0x7f9dd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9dd006ae70 0x7f9dd006d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9de0036000 tx=0x7f9de003c040 comp rx=0 tx=0).stop 2023-11-24T06:29:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013e360 msgr2=0x7f9df0130600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 --2- 172.21.15.64:0/1283509373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df013e360 0x7f9df0130600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9de8007d80 tx=0x7f9de800e4b0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9deffff640 1 -- 172.21.15.64:0/1283509373 reap_dead start 2023-11-24T06:29:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 shutdown_connections 2023-11-24T06:29:49.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 >> 172.21.15.64:0/1283509373 conn(0x7f9df006a800 msgr2=0x7f9df00af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 shutdown_connections 2023-11-24T06:29:49.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.054+0000 7f9df5b53640 1 -- 172.21.15.64:0/1283509373 wait complete. 2023-11-24T06:29:49.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T06:29:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.278+0000 7f9f2759e640 1 Processor -- start 2023-11-24T06:29:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.278+0000 7f9f2759e640 1 -- start start 2023-11-24T06:29:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.280+0000 7f9f2759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9f2815efa0 0x7f9f28071180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.280+0000 7f9f2759e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9f2815f380 0x7f9f280b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.280+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f281628b0 0x7f9f280b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.280+0000 7f9f2759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280b2db0 con 0x7f9f2815efa0 2023-11-24T06:29:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.280+0000 7f9f2759e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280b1ff0 con 0x7f9f2815f380 2023-11-24T06:29:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.280+0000 7f9f2759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280b22d0 con 0x7f9f281628b0 2023-11-24T06:29:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.280+0000 7f9f2659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9f2815efa0 0x7f9f28071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f2659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9f2815efa0 0x7f9f28071180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46090/0 (socket says 172.21.15.64:46090) 2023-11-24T06:29:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f2659c640 1 -- 172.21.15.64:0/3092802382 learned_addr learned my addr 172.21.15.64:0/3092802382 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f1ffff640 1 --2- 172.21.15.64:0/3092802382 >> v2:172.21.15.162:3300/0 conn(0x7f9f2815f380 0x7f9f280b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f26d9d640 1 --2- 172.21.15.64:0/3092802382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f281628b0 0x7f9f280b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f2659c640 1 -- 172.21.15.64:0/3092802382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f281628b0 msgr2=0x7f9f280b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f2659c640 1 --2- 172.21.15.64:0/3092802382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f281628b0 0x7f9f280b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f2659c640 1 -- 172.21.15.64:0/3092802382 >> v2:172.21.15.162:3300/0 conn(0x7f9f2815f380 msgr2=0x7f9f280b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f2659c640 1 --2- 172.21.15.64:0/3092802382 >> v2:172.21.15.162:3300/0 conn(0x7f9f2815f380 0x7f9f280b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f2659c640 1 -- 172.21.15.64:0/3092802382 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280b25b0 con 0x7f9f2815efa0 2023-11-24T06:29:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f26d9d640 1 --2- 172.21.15.64:0/3092802382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f281628b0 0x7f9f280b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.281+0000 7f9f1ffff640 1 --2- 172.21.15.64:0/3092802382 >> v2:172.21.15.162:3300/0 conn(0x7f9f2815f380 0x7f9f280b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.282+0000 7f9f2659c640 1 --2- 172.21.15.64:0/3092802382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2815efa0 0x7f9f28071180 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009870 tx=0x7f9f1802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41865fafb3cfc10a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.282+0000 7f9f25d9b640 1 -- 172.21.15.64:0/3092802382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18031640 con 0x7f9f2815efa0 2023-11-24T06:29:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f25d9b640 1 -- 172.21.15.64:0/3092802382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180317a0 con 0x7f9f2815efa0 2023-11-24T06:29:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f25d9b640 1 -- 172.21.15.64:0/3092802382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18031b30 con 0x7f9f2815efa0 2023-11-24T06:29:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 -- 172.21.15.64:0/3092802382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2815efa0 msgr2=0x7f9f28071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 --2- 172.21.15.64:0/3092802382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2815efa0 0x7f9f28071180 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009870 tx=0x7f9f1802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 -- 172.21.15.64:0/3092802382 shutdown_connections 2023-11-24T06:29:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 --2- 172.21.15.64:0/3092802382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f281628b0 0x7f9f280b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 --2- 172.21.15.64:0/3092802382 >> v2:172.21.15.162:3300/0 conn(0x7f9f2815f380 0x7f9f280b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 --2- 172.21.15.64:0/3092802382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2815efa0 0x7f9f28071180 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 -- 172.21.15.64:0/3092802382 >> 172.21.15.64:0/3092802382 conn(0x7f9f2806a3e0 msgr2=0x7f9f2806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 -- 172.21.15.64:0/3092802382 shutdown_connections 2023-11-24T06:29:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.283+0000 7f9f2759e640 1 -- 172.21.15.64:0/3092802382 wait complete. 2023-11-24T06:29:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.285+0000 7f9f2759e640 1 Processor -- start 2023-11-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.285+0000 7f9f2759e640 1 -- start start 2023-11-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f2815efa0 0x7f9f2814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2815f380 0x7f9f2814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f281628b0 0x7f9f280aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f2759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280a8870 con 0x7f9f281628b0 2023-11-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f2759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2815b620 con 0x7f9f2815f380 2023-11-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f2759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280cc4b0 con 0x7f9f2815efa0 2023-11-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f281628b0 0x7f9f280aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f281628b0 0x7f9f280aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46100/0 (socket says 172.21.15.64:46100) 2023-11-24T06:29:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.286+0000 7f9f26d9d640 1 -- 172.21.15.64:0/53725088 learned_addr learned my addr 172.21.15.64:0/53725088 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f26d9d640 1 -- 172.21.15.64:0/53725088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f2815efa0 msgr2=0x7f9f2814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f26d9d640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f2815efa0 0x7f9f2814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f26d9d640 1 -- 172.21.15.64:0/53725088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2815f380 msgr2=0x7f9f2814bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f26d9d640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2815f380 0x7f9f2814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f26d9d640 1 -- 172.21.15.64:0/53725088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180094d0 con 0x7f9f281628b0 2023-11-24T06:29:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f26d9d640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f281628b0 0x7f9f280aaaa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000cd80 tx=0x7f9f2000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f1effd640 1 -- 172.21.15.64:0/53725088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000f680 con 0x7f9f281628b0 2023-11-24T06:29:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f1effd640 1 -- 172.21.15.64:0/53725088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2000f800 con 0x7f9f281628b0 2023-11-24T06:29:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f1effd640 1 -- 172.21.15.64:0/53725088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000fbb0 con 0x7f9f281628b0 2023-11-24T06:29:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f280ae1f0 con 0x7f9f281628b0 2023-11-24T06:29:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.287+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f280cc9e0 con 0x7f9f281628b0 2023-11-24T06:29:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.289+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f28067450 con 0x7f9f281628b0 2023-11-24T06:29:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.289+0000 7f9f1effd640 1 -- 172.21.15.64:0/53725088 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f2000b040 con 0x7f9f281628b0 2023-11-24T06:29:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.291+0000 7f9f1effd640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f0806ae40 0x7f9f0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.291+0000 7f9f1effd640 1 -- 172.21.15.64:0/53725088 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9f20089c80 con 0x7f9f281628b0 2023-11-24T06:29:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.291+0000 7f9f2659c640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f0806ae40 0x7f9f0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.292+0000 7f9f2659c640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f0806ae40 0x7f9f0806d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f18036000 tx=0x7f9f18030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.295+0000 7f9f1effd640 1 -- 172.21.15.64:0/53725088 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f20055020 con 0x7f9f281628b0 2023-11-24T06:29:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.489+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f28165180 con 0x7f9f0806ae40 2023-11-24T06:29:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.490+0000 7f9f1effd640 1 -- 172.21.15.64:0/53725088 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9f18031a10 con 0x7f9f0806ae40 2023-11-24T06:29:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T06:29:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f0806ae40 msgr2=0x7f9f0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f0806ae40 0x7f9f0806d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f18036000 tx=0x7f9f18030000 comp rx=0 tx=0).stop 2023-11-24T06:29:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f281628b0 msgr2=0x7f9f280aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f281628b0 0x7f9f280aaaa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000cd80 tx=0x7f9f2000a650 comp rx=0 tx=0).stop 2023-11-24T06:29:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 shutdown_connections 2023-11-24T06:29:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9f0806ae40 0x7f9f0806d300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f281628b0 0x7f9f280aaaa0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2815f380 0x7f9f2814bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 --2- 172.21.15.64:0/53725088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f2815efa0 0x7f9f2814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.492+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 >> 172.21.15.64:0/53725088 conn(0x7f9f2806a3e0 msgr2=0x7f9f2815f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.493+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 shutdown_connections 2023-11-24T06:29:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.493+0000 7f9f2759e640 1 -- 172.21.15.64:0/53725088 wait complete. 2023-11-24T06:29:49.513 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T06:29:49.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:49.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.743+0000 7f8219213640 1 Processor -- start 2023-11-24T06:29:49.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.743+0000 7f8219213640 1 -- start start 2023-11-24T06:29:49.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.744+0000 7f8219213640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82140cc500 0x7f82140cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.744+0000 7f8219213640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f821406d160 0x7f821406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.744+0000 7f8219213640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8214070e50 con 0x7f8214143c40 2023-11-24T06:29:49.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.745+0000 7f8219213640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82140723c0 con 0x7f82140cc500 2023-11-24T06:29:49.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.745+0000 7f8219213640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82140cced0 con 0x7f821406d160 2023-11-24T06:29:49.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.745+0000 7f820bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82140cc500 0x7f82140cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.745+0000 7f820bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82140cc500 0x7f82140cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46114/0 (socket says 172.21.15.64:46114) 2023-11-24T06:29:49.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.745+0000 7f820bfff640 1 -- 172.21.15.64:0/1745610788 learned_addr learned my addr 172.21.15.64:0/1745610788 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:49.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.745+0000 7f8218a12640 1 --2- 172.21.15.64:0/1745610788 >> v2:172.21.15.162:3300/0 conn(0x7f821406d160 0x7f821406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.745+0000 7f820bfff640 1 -- 172.21.15.64:0/1745610788 >> v2:172.21.15.162:3300/0 conn(0x7f821406d160 msgr2=0x7f821406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.746+0000 7f820bfff640 1 --2- 172.21.15.64:0/1745610788 >> v2:172.21.15.162:3300/0 conn(0x7f821406d160 0x7f821406d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.746+0000 7f820bfff640 1 -- 172.21.15.64:0/1745610788 >> v1:172.21.15.162:6789/0 conn(0x7f8214143c40 legacy=0x7f8214071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.746+0000 7f820bfff640 1 -- 172.21.15.64:0/1745610788 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214144020 con 0x7f82140cc500 2023-11-24T06:29:49.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.747+0000 7f820bfff640 1 --2- 172.21.15.64:0/1745610788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82140cc500 0x7f82140cc990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8214070690 tx=0x7f81fc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de40b1f6d9c84add server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.747+0000 7f82137fe640 1 -- 172.21.15.64:0/1745610788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc031800 con 0x7f82140cc500 2023-11-24T06:29:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.747+0000 7f82137fe640 1 -- 172.21.15.64:0/1745610788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc031960 con 0x7f82140cc500 2023-11-24T06:29:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.747+0000 7f82137fe640 1 -- 172.21.15.64:0/1745610788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc031cf0 con 0x7f82140cc500 2023-11-24T06:29:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 -- 172.21.15.64:0/1745610788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82140cc500 msgr2=0x7f82140cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 --2- 172.21.15.64:0/1745610788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82140cc500 0x7f82140cc990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8214070690 tx=0x7f81fc02b8a0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 -- 172.21.15.64:0/1745610788 shutdown_connections 2023-11-24T06:29:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 --2- 172.21.15.64:0/1745610788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82140cc500 0x7f82140cc990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 --2- 172.21.15.64:0/1745610788 >> v2:172.21.15.162:3300/0 conn(0x7f821406d160 0x7f821406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 -- 172.21.15.64:0/1745610788 >> 172.21.15.64:0/1745610788 conn(0x7f821406a800 msgr2=0x7f821406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 -- 172.21.15.64:0/1745610788 shutdown_connections 2023-11-24T06:29:49.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.748+0000 7f8219213640 1 -- 172.21.15.64:0/1745610788 wait complete. 2023-11-24T06:29:49.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.750+0000 7f8219213640 1 Processor -- start 2023-11-24T06:29:49.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.750+0000 7f8219213640 1 -- start start 2023-11-24T06:29:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8219213640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f821406d160 0x7f821413d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8219213640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82140cc500 0x7f821413df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8219213640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8214143c40 0x7f821413e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8219213640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82140ad550 con 0x7f8214143c40 2023-11-24T06:29:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8219213640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82140ad830 con 0x7f821406d160 2023-11-24T06:29:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8219213640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82140adb10 con 0x7f82140cc500 2023-11-24T06:29:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8218a12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8214143c40 0x7f821413e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8218a12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8214143c40 0x7f821413e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46118/0 (socket says 172.21.15.64:46118) 2023-11-24T06:29:49.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8218a12640 1 -- 172.21.15.64:0/1250280663 learned_addr learned my addr 172.21.15.64:0/1250280663 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8213fff640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f821406d160 0x7f821413d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f820bfff640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82140cc500 0x7f821413df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8218a12640 1 -- 172.21.15.64:0/1250280663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82140cc500 msgr2=0x7f821413df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.751+0000 7f8218a12640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82140cc500 0x7f821413df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f8218a12640 1 -- 172.21.15.64:0/1250280663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f821406d160 msgr2=0x7f821413d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f8218a12640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f821406d160 0x7f821413d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f8218a12640 1 -- 172.21.15.64:0/1250280663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0094d0 con 0x7f8214143c40 2023-11-24T06:29:49.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f8218a12640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8214143c40 0x7f821413e450 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f820c007e40 tx=0x7f820c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c014620 con 0x7f8214143c40 2023-11-24T06:29:49.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c00ee90 con 0x7f8214143c40 2023-11-24T06:29:49.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c0149e0 con 0x7f8214143c40 2023-11-24T06:29:49.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82140addf0 con 0x7f8214143c40 2023-11-24T06:29:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.752+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821413dac0 con 0x7f8214143c40 2023-11-24T06:29:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.753+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f821406c960 con 0x7f8214143c40 2023-11-24T06:29:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.754+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f820c014b40 con 0x7f8214143c40 2023-11-24T06:29:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.755+0000 7f82117fa640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f81f406ae70 0x7f81f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.755+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f820c089cc0 con 0x7f8214143c40 2023-11-24T06:29:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.756+0000 7f82117fa640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f81f4070cd0 0x7f81f40730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.756+0000 7f8213fff640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f81f406ae70 0x7f81f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.756+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 --> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81f40737a0 con 0x7f81f4070cd0 2023-11-24T06:29:49.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.756+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820c08e030 con 0x7f8214143c40 2023-11-24T06:29:49.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.756+0000 7f820bfff640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f81f4070cd0 0x7f81f40730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.756+0000 7f8213fff640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f81f406ae70 0x7f81f406d330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f820400a750 tx=0x7f8204008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.757+0000 7f820bfff640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f81f4070cd0 0x7f81f40730f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.757+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== osd.3 v2:172.21.15.64:6800/3744810476 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f81f40737a0 con 0x7f81f4070cd0 2023-11-24T06:29:49.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.772+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 --> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f82140720e0 con 0x7f81f4070cd0 2023-11-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f82117fa640 1 -- 172.21.15.64:0/1250280663 <== osd.3 v2:172.21.15.64:6800/3744810476 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f81fc031b50 con 0x7f81f4070cd0 2023-11-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:49.776 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:49.776 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f81f4070cd0 msgr2=0x7f81f40730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6800/3744810476,v1:172.21.15.64:6801/3744810476] conn(0x7f81f4070cd0 0x7f81f40730f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f81f406ae70 msgr2=0x7f81f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f81f406ae70 0x7f81f406d330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f820400a750 tx=0x7f8204008040 comp rx=0 tx=0).stop 2023-11-24T06:29:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8214143c40 msgr2=0x7f821413e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 --2- 172.21.15.64:0/1250280663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8214143c40 0x7f821413e450 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f820c007e40 tx=0x7f820c00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8218a12640 1 -- 172.21.15.64:0/1250280663 reap_dead start 2023-11-24T06:29:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 shutdown_connections 2023-11-24T06:29:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 >> 172.21.15.64:0/1250280663 conn(0x7f821406a800 msgr2=0x7f8214071290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 shutdown_connections 2023-11-24T06:29:49.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:49.773+0000 7f8219213640 1 -- 172.21.15.64:0/1250280663 wait complete. 2023-11-24T06:29:49.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T06:29:50.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.008+0000 7feaeaa7e640 1 Processor -- start 2023-11-24T06:29:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.008+0000 7feaeaa7e640 1 -- start start 2023-11-24T06:29:50.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaeaa7e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 0x7feae40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaeaa7e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feae406dab0 0x7feae40ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaeaa7e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae406ca10 con 0x7feae41447f0 2023-11-24T06:29:50.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaeaa7e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae406d690 con 0x7feae406dab0 2023-11-24T06:29:50.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaeaa7e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40cebb0 con 0x7feae406c470 2023-11-24T06:29:50.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaea27d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feae406dab0 0x7feae40ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaea27d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feae406dab0 0x7feae40ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46124/0 (socket says 172.21.15.64:46124) 2023-11-24T06:29:50.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.010+0000 7feaea27d640 1 -- 172.21.15.64:0/325836747 learned_addr learned my addr 172.21.15.64:0/325836747 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:50.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.011+0000 7feaea27d640 1 -- 172.21.15.64:0/325836747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 msgr2=0x7feae40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:50.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.011+0000 7feaea27d640 1 --2- 172.21.15.64:0/325836747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 0x7feae40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.011+0000 7feaea27d640 1 -- 172.21.15.64:0/325836747 >> v1:172.21.15.162:6789/0 conn(0x7feae41447f0 legacy=0x7feae415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:50.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.011+0000 7feaea27d640 1 -- 172.21.15.64:0/325836747 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae406ccf0 con 0x7feae406dab0 2023-11-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.012+0000 7feaea27d640 1 --2- 172.21.15.64:0/325836747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae406dab0 0x7feae40ccb20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feadc009e50 tx=0x7feadc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d76fd3087f6ba0c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.012+0000 7feae8a7a640 1 -- 172.21.15.64:0/325836747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc02c9e0 con 0x7feae406dab0 2023-11-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.012+0000 7feae8a7a640 1 -- 172.21.15.64:0/325836747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc02cb40 con 0x7feae406dab0 2023-11-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.012+0000 7feae8a7a640 1 -- 172.21.15.64:0/325836747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0343f0 con 0x7feae406dab0 2023-11-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 -- 172.21.15.64:0/325836747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae406dab0 msgr2=0x7feae40ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/325836747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae406dab0 0x7feae40ccb20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feadc009e50 tx=0x7feadc02be40 comp rx=0 tx=0).stop 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 -- 172.21.15.64:0/325836747 shutdown_connections 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/325836747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae406dab0 0x7feae40ccb20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/325836747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 0x7feae40cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 -- 172.21.15.64:0/325836747 >> 172.21.15.64:0/325836747 conn(0x7feae406a3e0 msgr2=0x7feae406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 -- 172.21.15.64:0/325836747 shutdown_connections 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.013+0000 7feaeaa7e640 1 -- 172.21.15.64:0/325836747 wait complete. 2023-11-24T06:29:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.015+0000 7feaeaa7e640 1 Processor -- start 2023-11-24T06:29:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.015+0000 7feaeaa7e640 1 -- start start 2023-11-24T06:29:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.016+0000 7feaeaa7e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 0x7feae41580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.016+0000 7feaeaa7e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae406dab0 0x7feae41391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.016+0000 7feaeaa7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae41447f0 0x7feae40ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.016+0000 7feaeaa7e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4070e30 con 0x7feae41447f0 2023-11-24T06:29:50.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.016+0000 7feaeaa7e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4070220 con 0x7feae406dab0 2023-11-24T06:29:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.016+0000 7feaeaa7e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae4070500 con 0x7feae406c470 2023-11-24T06:29:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae41447f0 0x7feae40ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae41447f0 0x7feae40ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46138/0 (socket says 172.21.15.64:46138) 2023-11-24T06:29:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 -- 172.21.15.64:0/1334536572 learned_addr learned my addr 172.21.15.64:0/1334536572 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feae9a7c640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 0x7feae41580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feae927b640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae406dab0 0x7feae41391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 -- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 msgr2=0x7feae41580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 0x7feae41580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 -- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae406dab0 msgr2=0x7feae41391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae406dab0 0x7feae41391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feaea27d640 1 -- 172.21.15.64:0/1334536572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009b30 con 0x7feae41447f0 2023-11-24T06:29:50.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.017+0000 7feae927b640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae406dab0 0x7feae41391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:50.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.018+0000 7feaea27d640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae41447f0 0x7feae40ad950 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feadc036040 tx=0x7feadc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.018+0000 7feada7fc640 1 -- 172.21.15.64:0/1334536572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc03e450 con 0x7feae41447f0 2023-11-24T06:29:50.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.018+0000 7feada7fc640 1 -- 172.21.15.64:0/1334536572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc03e5d0 con 0x7feae41447f0 2023-11-24T06:29:50.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.018+0000 7feada7fc640 1 -- 172.21.15.64:0/1334536572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc03e980 con 0x7feae41447f0 2023-11-24T06:29:50.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.018+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae40707e0 con 0x7feae41447f0 2023-11-24T06:29:50.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.018+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae41585f0 con 0x7feae41447f0 2023-11-24T06:29:50.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.019+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4070fe0 con 0x7feae41447f0 2023-11-24T06:29:50.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.020+0000 7feada7fc640 1 -- 172.21.15.64:0/1334536572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feadc03eae0 con 0x7feae41447f0 2023-11-24T06:29:50.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.021+0000 7feada7fc640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7feac806ae40 0x7feac806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.021+0000 7feada7fc640 1 -- 172.21.15.64:0/1334536572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7feadc0ac210 con 0x7feae41447f0 2023-11-24T06:29:50.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.021+0000 7feae9a7c640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7feac806ae40 0x7feac806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.022+0000 7feae9a7c640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7feac806ae40 0x7feac806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feae0006fd0 tx=0x7feae0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.025+0000 7feada7fc640 1 -- 172.21.15.64:0/1334536572 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feae4070fe0 con 0x7feae41447f0 2023-11-24T06:29:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.227+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7feae4070670 con 0x7feac806ae40 2023-11-24T06:29:50.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.228+0000 7feada7fc640 1 -- 172.21.15.64:0/1334536572 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7feae4070670 con 0x7feac806ae40 2023-11-24T06:29:50.231 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T06:29:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7feac806ae40 msgr2=0x7feac806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7feac806ae40 0x7feac806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feae0006fd0 tx=0x7feae0008040 comp rx=0 tx=0).stop 2023-11-24T06:29:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae41447f0 msgr2=0x7feae40ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae41447f0 0x7feae40ad950 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feadc036040 tx=0x7feadc004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 shutdown_connections 2023-11-24T06:29:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7feac806ae40 0x7feac806d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae41447f0 0x7feae40ad950 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae406dab0 0x7feae41391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 --2- 172.21.15.64:0/1334536572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feae406c470 0x7feae41580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 >> 172.21.15.64:0/1334536572 conn(0x7feae406a3e0 msgr2=0x7feae4159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 shutdown_connections 2023-11-24T06:29:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.231+0000 7feaeaa7e640 1 -- 172.21.15.64:0/1334536572 wait complete. 2023-11-24T06:29:50.254 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T06:29:50.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:50.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.481+0000 7fc5b319e640 1 Processor -- start 2023-11-24T06:29:50.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.481+0000 7fc5b319e640 1 -- start start 2023-11-24T06:29:50.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.483+0000 7fc5b319e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc5ac160920 0x7fc5ac0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.483+0000 7fc5b319e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac0d0a10 con 0x7fc5ac0cfea0 2023-11-24T06:29:50.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.483+0000 7fc5b319e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac161e30 con 0x7fc5ac0cf9f0 2023-11-24T06:29:50.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.483+0000 7fc5b319e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0d1f50 con 0x7fc5ac160920 2023-11-24T06:29:50.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.483+0000 7fc5b219c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc5ac160920 0x7fc5ac0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.483+0000 7fc5b299d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc5ac0cfea0 0x7fc5ac161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35876/0 (socket says 172.21.15.64:35876) 2023-11-24T06:29:50.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.483+0000 7fc5b299d640 1 -- 172.21.15.64:0/271718329 learned_addr learned my addr 172.21.15.64:0/271718329 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:50.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.484+0000 7fc5b219c640 1 -- 172.21.15.64:0/271718329 >> v1:172.21.15.162:6789/0 conn(0x7fc5ac0cf9f0 legacy=0x7fc5ac161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:50.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.484+0000 7fc5b219c640 1 -- 172.21.15.64:0/271718329 >> v1:172.21.15.64:6789/0 conn(0x7fc5ac0cfea0 legacy=0x7fc5ac161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.484+0000 7fc5b119a640 1 -- 172.21.15.64:0/271718329 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028661919 0 0) 0x7fc5ac0d0a10 con 0x7fc5ac0cfea0 2023-11-24T06:29:50.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.484+0000 7fc5b219c640 1 -- 172.21.15.64:0/271718329 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0cf080 con 0x7fc5ac160920 2023-11-24T06:29:50.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.485+0000 7fc5b219c640 1 --2- 172.21.15.64:0/271718329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b3f30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80097a0 tx=0x7fc5a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1351be7dea1bdfe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b119a640 1 -- 172.21.15.64:0/271718329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8031830 con 0x7fc5ac160920 2023-11-24T06:29:50.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b119a640 1 -- 172.21.15.64:0/271718329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a8031990 con 0x7fc5ac160920 2023-11-24T06:29:50.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b119a640 1 -- 172.21.15.64:0/271718329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8031d20 con 0x7fc5ac160920 2023-11-24T06:29:50.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b319e640 1 -- 172.21.15.64:0/271718329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 msgr2=0x7fc5ac0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b319e640 1 --2- 172.21.15.64:0/271718329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b3f30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80097a0 tx=0x7fc5a802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b319e640 1 -- 172.21.15.64:0/271718329 shutdown_connections 2023-11-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b319e640 1 --2- 172.21.15.64:0/271718329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b3f30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b319e640 1 -- 172.21.15.64:0/271718329 >> 172.21.15.64:0/271718329 conn(0x7fc5ac0d3090 msgr2=0x7fc5ac0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.486+0000 7fc5b319e640 1 -- 172.21.15.64:0/271718329 shutdown_connections 2023-11-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.487+0000 7fc5b319e640 1 -- 172.21.15.64:0/271718329 wait complete. 2023-11-24T06:29:50.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.488+0000 7fc5b319e640 1 Processor -- start 2023-11-24T06:29:50.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.488+0000 7fc5b319e640 1 -- start start 2023-11-24T06:29:50.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.489+0000 7fc5b319e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0cf9f0 0x7fc5ac0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.489+0000 7fc5b319e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac0cfea0 0x7fc5ac1f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.489+0000 7fc5b319e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.489+0000 7fc5b319e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0caa40 con 0x7fc5ac160920 2023-11-24T06:29:50.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.489+0000 7fc5b319e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cad20 con 0x7fc5ac0cf9f0 2023-11-24T06:29:50.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.489+0000 7fc5b319e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cb000 con 0x7fc5ac0cfea0 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46158/0 (socket says 172.21.15.64:46158) 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 -- 172.21.15.64:0/2568951396 learned_addr learned my addr 172.21.15.64:0/2568951396 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b199b640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac0cfea0 0x7fc5ac1f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 -- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac0cfea0 msgr2=0x7fc5ac1f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac0cfea0 0x7fc5ac1f2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 -- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0cf9f0 msgr2=0x7fc5ac0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0cf9f0 0x7fc5ac0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 -- 172.21.15.64:0/2568951396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8009400 con 0x7fc5ac160920 2023-11-24T06:29:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.490+0000 7fc5b299d640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b8060 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400c7e0 tx=0x7fc5a400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.491+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400a640 con 0x7fc5ac160920 2023-11-24T06:29:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.491+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a400a7c0 con 0x7fc5ac160920 2023-11-24T06:29:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.491+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac0cb2e0 con 0x7fc5ac160920 2023-11-24T06:29:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.491+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400ab70 con 0x7fc5ac160920 2023-11-24T06:29:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.491+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac1f2b60 con 0x7fc5ac160920 2023-11-24T06:29:50.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.492+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc5ac0ce7e0 con 0x7fc5ac160920 2023-11-24T06:29:50.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.493+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5a400acd0 con 0x7fc5ac160920 2023-11-24T06:29:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.494+0000 7fc5a2ffd640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc59006ae40 0x7fc59006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.494+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc5a4088c70 con 0x7fc5ac160920 2023-11-24T06:29:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.495+0000 7fc5a2ffd640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fc590070ca0 0x7fc5900730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.495+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 --> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc590073770 con 0x7fc590070ca0 2023-11-24T06:29:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.495+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5a408d2a0 con 0x7fc5ac160920 2023-11-24T06:29:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.495+0000 7fc5b219c640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc59006ae40 0x7fc59006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.495+0000 7fc5b199b640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fc590070ca0 0x7fc5900730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.495+0000 7fc5b219c640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc59006ae40 0x7fc59006d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8036000 tx=0x7fc5a8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.496+0000 7fc5b199b640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fc590070ca0 0x7fc5900730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.498+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== osd.4 v2:172.21.15.162:6802/399042930 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc590073770 con 0x7fc590070ca0 2023-11-24T06:29:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.513+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 --> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc5ac0c6be0 con 0x7fc590070ca0 2023-11-24T06:29:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.514+0000 7fc5a2ffd640 1 -- 172.21.15.64:0/2568951396 <== osd.4 v2:172.21.15.162:6802/399042930 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc59c00a880 con 0x7fc590070ca0 2023-11-24T06:29:50.516 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:50.517 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:50.517 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.514+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fc590070ca0 msgr2=0x7fc5900730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.514+0000 7fc5b319e640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.162:6802/399042930,v1:172.21.15.162:6804/399042930] conn(0x7fc590070ca0 0x7fc5900730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc59006ae40 msgr2=0x7fc59006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc59006ae40 0x7fc59006d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8036000 tx=0x7fc5a8030000 comp rx=0 tx=0).stop 2023-11-24T06:29:50.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 msgr2=0x7fc5ac0b8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 --2- 172.21.15.64:0/2568951396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac160920 0x7fc5ac0b8060 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400c7e0 tx=0x7fc5a400cca0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b299d640 1 -- 172.21.15.64:0/2568951396 reap_dead start 2023-11-24T06:29:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 shutdown_connections 2023-11-24T06:29:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 >> 172.21.15.64:0/2568951396 conn(0x7fc5ac0d3090 msgr2=0x7fc5ac0b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 shutdown_connections 2023-11-24T06:29:50.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.515+0000 7fc5b319e640 1 -- 172.21.15.64:0/2568951396 wait complete. 2023-11-24T06:29:50.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T06:29:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.755+0000 7fce5e0d1640 1 Processor -- start 2023-11-24T06:29:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.755+0000 7fce5e0d1640 1 -- start start 2023-11-24T06:29:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.756+0000 7fce5e0d1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce58153c10 0x7fce580b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.756+0000 7fce5e0d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce5806d410 0x7fce580ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5e0d1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5815aba0 con 0x7fce580b3e40 2023-11-24T06:29:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5e0d1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce58159bf0 con 0x7fce5806d410 2023-11-24T06:29:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5e0d1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce58159de0 con 0x7fce58153c10 2023-11-24T06:29:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5d8d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce5806d410 0x7fce580ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5d8d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce5806d410 0x7fce580ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46166/0 (socket says 172.21.15.64:46166) 2023-11-24T06:29:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5d8d0640 1 -- 172.21.15.64:0/1001880461 learned_addr learned my addr 172.21.15.64:0/1001880461 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5c8ce640 1 --1- 172.21.15.64:0/1001880461 >> v1:172.21.15.64:6789/0 conn(0x7fce580b3e40 0x7fce5806cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35882/0 (socket says 172.21.15.64:35882) 2023-11-24T06:29:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5d0cf640 1 --2- 172.21.15.64:0/1001880461 >> v2:172.21.15.162:3300/0 conn(0x7fce58153c10 0x7fce580b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5d8d0640 1 -- 172.21.15.64:0/1001880461 >> v2:172.21.15.162:3300/0 conn(0x7fce58153c10 msgr2=0x7fce580b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5d8d0640 1 --2- 172.21.15.64:0/1001880461 >> v2:172.21.15.162:3300/0 conn(0x7fce58153c10 0x7fce580b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce5d8d0640 1 -- 172.21.15.64:0/1001880461 >> v1:172.21.15.64:6789/0 conn(0x7fce580b3e40 legacy=0x7fce5806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.758+0000 7fce5d8d0640 1 -- 172.21.15.64:0/1001880461 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5815a0c0 con 0x7fce5806d410 2023-11-24T06:29:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.758+0000 7fce5d0cf640 1 --2- 172.21.15.64:0/1001880461 >> v2:172.21.15.162:3300/0 conn(0x7fce58153c10 0x7fce580b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.757+0000 7fce4ffff640 1 -- 172.21.15.64:0/1001880461 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408043940 0 0) 0x7fce5815aba0 con 0x7fce580b3e40 2023-11-24T06:29:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.759+0000 7fce5d8d0640 1 --2- 172.21.15.64:0/1001880461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5806d410 0x7fce580ccd90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fce5806e330 tx=0x7fce5002e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce875050f1465de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.759+0000 7fce4ffff640 1 -- 172.21.15.64:0/1001880461 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50038400 con 0x7fce5806d410 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.759+0000 7fce4ffff640 1 -- 172.21.15.64:0/1001880461 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50032070 con 0x7fce5806d410 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.759+0000 7fce4ffff640 1 -- 172.21.15.64:0/1001880461 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50038780 con 0x7fce5806d410 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.759+0000 7fce5e0d1640 1 -- 172.21.15.64:0/1001880461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5806d410 msgr2=0x7fce580ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.759+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/1001880461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5806d410 0x7fce580ccd90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fce5806e330 tx=0x7fce5002e180 comp rx=0 tx=0).stop 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.760+0000 7fce5e0d1640 1 -- 172.21.15.64:0/1001880461 shutdown_connections 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.760+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/1001880461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5806d410 0x7fce580ccd90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.760+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/1001880461 >> v2:172.21.15.162:3300/0 conn(0x7fce58153c10 0x7fce580b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.760+0000 7fce5e0d1640 1 -- 172.21.15.64:0/1001880461 >> 172.21.15.64:0/1001880461 conn(0x7fce5806a3e0 msgr2=0x7fce5806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.760+0000 7fce5e0d1640 1 -- 172.21.15.64:0/1001880461 shutdown_connections 2023-11-24T06:29:50.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.760+0000 7fce5e0d1640 1 -- 172.21.15.64:0/1001880461 wait complete. 2023-11-24T06:29:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.761+0000 7fce5e0d1640 1 Processor -- start 2023-11-24T06:29:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.762+0000 7fce5e0d1640 1 -- start start 2023-11-24T06:29:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5e0d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce5806d410 0x7fce580a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5e0d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce580b3e40 0x7fce581445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5e0d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce58153c10 0x7fce58158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5e0d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58146a70 con 0x7fce580b3e40 2023-11-24T06:29:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5e0d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58146d10 con 0x7fce58153c10 2023-11-24T06:29:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5e0d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58146ff0 con 0x7fce5806d410 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce580b3e40 0x7fce581445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce580b3e40 0x7fce581445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46174/0 (socket says 172.21.15.64:46174) 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 -- 172.21.15.64:0/50686587 learned_addr learned my addr 172.21.15.64:0/50686587 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 -- 172.21.15.64:0/50686587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce5806d410 msgr2=0x7fce580a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce5806d410 0x7fce580a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 -- 172.21.15.64:0/50686587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce58153c10 msgr2=0x7fce58158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce58153c10 0x7fce58158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.763+0000 7fce5c8ce640 1 -- 172.21.15.64:0/50686587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5000bd40 con 0x7fce580b3e40 2023-11-24T06:29:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.764+0000 7fce5c8ce640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce580b3e40 0x7fce581445b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c4b0 tx=0x7fce4800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.764+0000 7fce4dffb640 1 -- 172.21.15.64:0/50686587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480074e0 con 0x7fce580b3e40 2023-11-24T06:29:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.764+0000 7fce4dffb640 1 -- 172.21.15.64:0/50686587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48007660 con 0x7fce580b3e40 2023-11-24T06:29:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.764+0000 7fce4dffb640 1 -- 172.21.15.64:0/50686587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48004a20 con 0x7fce580b3e40 2023-11-24T06:29:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.764+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce581472d0 con 0x7fce580b3e40 2023-11-24T06:29:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.764+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce58144af0 con 0x7fce580b3e40 2023-11-24T06:29:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.765+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce58147920 con 0x7fce580b3e40 2023-11-24T06:29:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.766+0000 7fce4dffb640 1 -- 172.21.15.64:0/50686587 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce48004b80 con 0x7fce580b3e40 2023-11-24T06:29:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.767+0000 7fce4dffb640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce3c06ae40 0x7fce3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.767+0000 7fce4dffb640 1 -- 172.21.15.64:0/50686587 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fce48088450 con 0x7fce580b3e40 2023-11-24T06:29:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.768+0000 7fce5d0cf640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce3c06ae40 0x7fce3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.768+0000 7fce5d0cf640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce3c06ae40 0x7fce3c06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce54006fd0 tx=0x7fce54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.772+0000 7fce4dffb640 1 -- 172.21.15.64:0/50686587 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce48053950 con 0x7fce580b3e40 2023-11-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.965+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fce58153ff0 con 0x7fce3c06ae40 2023-11-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.966+0000 7fce4dffb640 1 -- 172.21.15.64:0/50686587 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fce5400c7a0 con 0x7fce3c06ae40 2023-11-24T06:29:50.969 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce3c06ae40 msgr2=0x7fce3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce3c06ae40 0x7fce3c06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce54006fd0 tx=0x7fce54008040 comp rx=0 tx=0).stop 2023-11-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce580b3e40 msgr2=0x7fce581445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce580b3e40 0x7fce581445b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c4b0 tx=0x7fce4800c970 comp rx=0 tx=0).stop 2023-11-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 shutdown_connections 2023-11-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fce3c06ae40 0x7fce3c06d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce58153c10 0x7fce58158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce580b3e40 0x7fce581445b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 --2- 172.21.15.64:0/50686587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce5806d410 0x7fce580a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 >> 172.21.15.64:0/50686587 conn(0x7fce5806a3e0 msgr2=0x7fce5806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 shutdown_connections 2023-11-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:50.969+0000 7fce5e0d1640 1 -- 172.21.15.64:0/50686587 wait complete. 2023-11-24T06:29:50.991 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T06:29:50.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.219+0000 7f9ff1f47640 1 Processor -- start 2023-11-24T06:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.220+0000 7f9ff1f47640 1 -- start start 2023-11-24T06:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.221+0000 7f9ff1f47640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9fec143c40 0x7f9fec071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.221+0000 7f9ff1f47640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9fec0717d0 0x7f9fec0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.221+0000 7f9ff1f47640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 0x7f9fec15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.221+0000 7f9ff1f47640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec0723c0 con 0x7f9fec0717d0 2023-11-24T06:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.221+0000 7f9ff1f47640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec070e50 con 0x7f9fec143c40 2023-11-24T06:29:51.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.221+0000 7f9ff1f47640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec0cce40 con 0x7f9fec0cca40 2023-11-24T06:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9fec0717d0 0x7f9fec0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9fec0717d0 0x7f9fec0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46180/0 (socket says 172.21.15.64:46180) 2023-11-24T06:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 -- 172.21.15.64:0/200927225 learned_addr learned my addr 172.21.15.64:0/200927225 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9ff0f45640 1 --2- 172.21.15.64:0/200927225 >> v2:172.21.15.162:3300/0 conn(0x7f9fec143c40 0x7f9fec071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9ff1746640 1 --2- 172.21.15.64:0/200927225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 0x7f9fec15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 -- 172.21.15.64:0/200927225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 msgr2=0x7f9fec15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 --2- 172.21.15.64:0/200927225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 0x7f9fec15eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 -- 172.21.15.64:0/200927225 >> v2:172.21.15.162:3300/0 conn(0x7f9fec143c40 msgr2=0x7f9fec071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 --2- 172.21.15.64:0/200927225 >> v2:172.21.15.162:3300/0 conn(0x7f9fec143c40 0x7f9fec071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.222+0000 7f9febfff640 1 -- 172.21.15.64:0/200927225 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec144020 con 0x7f9fec0717d0 2023-11-24T06:29:51.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.223+0000 7f9febfff640 1 --2- 172.21.15.64:0/200927225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec0cc500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9fec06e410 tx=0x7f9fd802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb9ad34fb5193d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.224+0000 7f9feb7fe640 1 -- 172.21.15.64:0/200927225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80317e0 con 0x7f9fec0717d0 2023-11-24T06:29:51.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.224+0000 7f9feb7fe640 1 -- 172.21.15.64:0/200927225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd8031940 con 0x7f9fec0717d0 2023-11-24T06:29:51.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.224+0000 7f9feb7fe640 1 -- 172.21.15.64:0/200927225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8031cd0 con 0x7f9fec0717d0 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.224+0000 7f9ff1f47640 1 -- 172.21.15.64:0/200927225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 msgr2=0x7f9fec0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.224+0000 7f9ff1f47640 1 --2- 172.21.15.64:0/200927225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec0cc500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9fec06e410 tx=0x7f9fd802b880 comp rx=0 tx=0).stop 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.225+0000 7f9ff1f47640 1 -- 172.21.15.64:0/200927225 shutdown_connections 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.225+0000 7f9ff1f47640 1 --2- 172.21.15.64:0/200927225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 0x7f9fec15eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.225+0000 7f9ff1f47640 1 --2- 172.21.15.64:0/200927225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec0cc500 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.225+0000 7f9ff1f47640 1 --2- 172.21.15.64:0/200927225 >> v2:172.21.15.162:3300/0 conn(0x7f9fec143c40 0x7f9fec071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.225+0000 7f9ff1f47640 1 -- 172.21.15.64:0/200927225 >> 172.21.15.64:0/200927225 conn(0x7f9fec06a800 msgr2=0x7f9fec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.225+0000 7f9ff1f47640 1 -- 172.21.15.64:0/200927225 shutdown_connections 2023-11-24T06:29:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.225+0000 7f9ff1f47640 1 -- 172.21.15.64:0/200927225 wait complete. 2023-11-24T06:29:51.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.226+0000 7f9ff1f47640 1 Processor -- start 2023-11-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.227+0000 7f9ff1f47640 1 -- start start 2023-11-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.227+0000 7f9ff1f47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec1359d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.227+0000 7f9ff1f47640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 0x7f9fec06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.227+0000 7f9ff1f47640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fec143c40 0x7f9fec06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.227+0000 7f9ff1f47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec136ce0 con 0x7f9fec0717d0 2023-11-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.227+0000 7f9ff1f47640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1360d0 con 0x7f9fec143c40 2023-11-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.227+0000 7f9ff1f47640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1363b0 con 0x7f9fec0cca40 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec1359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec1359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46192/0 (socket says 172.21.15.64:46192) 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 -- 172.21.15.64:0/156053980 learned_addr learned my addr 172.21.15.64:0/156053980 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 -- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 msgr2=0x7f9fec06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9febfff640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 0x7f9fec06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff1746640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fec143c40 0x7f9fec06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fec0cca40 0x7f9fec06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 -- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fec143c40 msgr2=0x7f9fec06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fec143c40 0x7f9fec06d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.228+0000 7f9ff0f45640 1 -- 172.21.15.64:0/156053980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80094d0 con 0x7f9fec0717d0 2023-11-24T06:29:51.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.229+0000 7f9ff0f45640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec1359d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000d800 tx=0x7f9fe000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.229+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000b6c0 con 0x7f9fec0717d0 2023-11-24T06:29:51.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.229+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe000b820 con 0x7f9fec0717d0 2023-11-24T06:29:51.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.229+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000bbb0 con 0x7f9fec0717d0 2023-11-24T06:29:51.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.229+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec136690 con 0x7f9fec0717d0 2023-11-24T06:29:51.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.229+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec15a280 con 0x7f9fec0717d0 2023-11-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.230+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9fec06c960 con 0x7f9fec0717d0 2023-11-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.231+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fe000bd10 con 0x7f9fec0717d0 2023-11-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.232+0000 7f9fe97fa640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9fd006ae40 0x7f9fd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.232+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9fe0089780 con 0x7f9fec0717d0 2023-11-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.232+0000 7f9febfff640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9fd006ae40 0x7f9fd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.233+0000 7f9fe97fa640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7f9fd0070ca0 0x7f9fd00730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.233+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 --> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fd0073770 con 0x7f9fd0070ca0 2023-11-24T06:29:51.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.233+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe008bde0 con 0x7f9fec0717d0 2023-11-24T06:29:51.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.233+0000 7f9febfff640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9fd006ae40 0x7f9fd006d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8036000 tx=0x7f9fd803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.233+0000 7f9ff1746640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7f9fd0070ca0 0x7f9fd00730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.234+0000 7f9ff1746640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7f9fd0070ca0 0x7f9fd00730c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.237+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== osd.5 v2:172.21.15.162:6807/694094717 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9fd0073770 con 0x7f9fd0070ca0 2023-11-24T06:29:51.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.251+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 --> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9fec0720e0 con 0x7f9fd0070ca0 2023-11-24T06:29:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.252+0000 7f9fe97fa640 1 -- 172.21.15.64:0/156053980 <== osd.5 v2:172.21.15.162:6807/694094717 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9fe4014380 con 0x7f9fd0070ca0 2023-11-24T06:29:51.256 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:51.256 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:51.256 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.252+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7f9fd0070ca0 msgr2=0x7f9fd00730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.252+0000 7f9ff1f47640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.162:6807/694094717,v1:172.21.15.162:6809/694094717] conn(0x7f9fd0070ca0 0x7f9fd00730c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9fd006ae40 msgr2=0x7f9fd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f9fd006ae40 0x7f9fd006d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8036000 tx=0x7f9fd803d040 comp rx=0 tx=0).stop 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 msgr2=0x7f9fec1359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 --2- 172.21.15.64:0/156053980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec0717d0 0x7f9fec1359d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000d800 tx=0x7f9fe000dcc0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1746640 1 -- 172.21.15.64:0/156053980 reap_dead start 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 shutdown_connections 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 >> 172.21.15.64:0/156053980 conn(0x7f9fec06a800 msgr2=0x7f9fec0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 shutdown_connections 2023-11-24T06:29:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.253+0000 7f9ff1f47640 1 -- 172.21.15.64:0/156053980 wait complete. 2023-11-24T06:29:51.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T06:29:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.502+0000 7fd6048ad640 1 Processor -- start 2023-11-24T06:29:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.502+0000 7fd6048ad640 1 -- start start 2023-11-24T06:29:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd6048ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd60015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd6048ad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd60015e1e0 0x7fd6000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd6048ad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6000b1ee0 con 0x7fd600143ae0 2023-11-24T06:29:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd6048ad640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6000b2780 con 0x7fd60015e1e0 2023-11-24T06:29:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd6048ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60015e990 con 0x7fd6000cf9a0 2023-11-24T06:29:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd5fed9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd60015e1e0 0x7fd6000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd5f7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd60015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd5fed9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd60015e1e0 0x7fd6000702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:36976/0 (socket says 172.21.15.64:36976) 2023-11-24T06:29:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.504+0000 7fd5fed9d640 1 -- 172.21.15.64:0/138502992 learned_addr learned my addr 172.21.15.64:0/138502992 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.505+0000 7fd5f7fff640 1 -- 172.21.15.64:0/138502992 >> v2:172.21.15.162:3300/0 conn(0x7fd60015e1e0 msgr2=0x7fd6000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.505+0000 7fd5f7fff640 1 --2- 172.21.15.64:0/138502992 >> v2:172.21.15.162:3300/0 conn(0x7fd60015e1e0 0x7fd6000702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.505+0000 7fd5f7fff640 1 -- 172.21.15.64:0/138502992 >> v1:172.21.15.162:6789/0 conn(0x7fd600143ae0 legacy=0x7fd6000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.505+0000 7fd5f7fff640 1 -- 172.21.15.64:0/138502992 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6000b2130 con 0x7fd6000cf9a0 2023-11-24T06:29:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.507+0000 7fd5f7fff640 1 --2- 172.21.15.64:0/138502992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd60015dca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80097a0 tx=0x7fd5e802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0f1ccd4c1b8a534 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.507+0000 7fd5fdd9b640 1 -- 172.21.15.64:0/138502992 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8031830 con 0x7fd6000cf9a0 2023-11-24T06:29:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.507+0000 7fd5fdd9b640 1 -- 172.21.15.64:0/138502992 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e8031990 con 0x7fd6000cf9a0 2023-11-24T06:29:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.507+0000 7fd5fdd9b640 1 -- 172.21.15.64:0/138502992 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8031d20 con 0x7fd6000cf9a0 2023-11-24T06:29:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 -- 172.21.15.64:0/138502992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 msgr2=0x7fd60015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 --2- 172.21.15.64:0/138502992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd60015dca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80097a0 tx=0x7fd5e802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 -- 172.21.15.64:0/138502992 shutdown_connections 2023-11-24T06:29:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 --2- 172.21.15.64:0/138502992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd60015dca0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 --2- 172.21.15.64:0/138502992 >> v2:172.21.15.162:3300/0 conn(0x7fd60015e1e0 0x7fd6000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 -- 172.21.15.64:0/138502992 >> 172.21.15.64:0/138502992 conn(0x7fd60006a800 msgr2=0x7fd60006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 -- 172.21.15.64:0/138502992 shutdown_connections 2023-11-24T06:29:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.508+0000 7fd6048ad640 1 -- 172.21.15.64:0/138502992 wait complete. 2023-11-24T06:29:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.510+0000 7fd6048ad640 1 Processor -- start 2023-11-24T06:29:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.510+0000 7fd6048ad640 1 -- start start 2023-11-24T06:29:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.511+0000 7fd6048ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd6000cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.511+0000 7fd6048ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600143ae0 0x7fd6000c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.511+0000 7fd6048ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd60015e1e0 0x7fd6000c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.511+0000 7fd6048ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000c6440 con 0x7fd60015e1e0 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.511+0000 7fd6048ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000c55b0 con 0x7fd600143ae0 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.511+0000 7fd6048ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000c4a30 con 0x7fd6000cf9a0 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.511+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd60015e1e0 0x7fd6000c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd60015e1e0 0x7fd6000c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46204/0 (socket says 172.21.15.64:46204) 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 -- 172.21.15.64:0/3492355256 learned_addr learned my addr 172.21.15.64:0/3492355256 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fe59c640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd6000cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5f7fff640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600143ae0 0x7fd6000c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 -- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 msgr2=0x7fd6000cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd6000cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 -- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600143ae0 msgr2=0x7fd6000c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600143ae0 0x7fd6000c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 -- 172.21.15.64:0/3492355256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8009400 con 0x7fd60015e1e0 2023-11-24T06:29:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5f7fff640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600143ae0 0x7fd6000c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.512+0000 7fd5fed9d640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd60015e1e0 0x7fd6000c4390 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009300 tx=0x7fd5f800e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.513+0000 7fd5f6ffd640 1 -- 172.21.15.64:0/3492355256 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8010040 con 0x7fd60015e1e0 2023-11-24T06:29:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.513+0000 7fd5f6ffd640 1 -- 172.21.15.64:0/3492355256 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f800eb20 con 0x7fd60015e1e0 2023-11-24T06:29:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.513+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6000c4d10 con 0x7fd60015e1e0 2023-11-24T06:29:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.513+0000 7fd5f6ffd640 1 -- 172.21.15.64:0/3492355256 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800eeb0 con 0x7fd60015e1e0 2023-11-24T06:29:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.513+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6000c52d0 con 0x7fd60015e1e0 2023-11-24T06:29:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.514+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6000675d0 con 0x7fd60015e1e0 2023-11-24T06:29:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.515+0000 7fd5f6ffd640 1 -- 172.21.15.64:0/3492355256 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5f8003990 con 0x7fd60015e1e0 2023-11-24T06:29:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.516+0000 7fd5f6ffd640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e006ae40 0x7fd5e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.516+0000 7fd5f6ffd640 1 -- 172.21.15.64:0/3492355256 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd5f80896f0 con 0x7fd60015e1e0 2023-11-24T06:29:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.516+0000 7fd5fe59c640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e006ae40 0x7fd5e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.517+0000 7fd5fe59c640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e006ae40 0x7fd5e006d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0000f60 tx=0x7fd5f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.520+0000 7fd5f6ffd640 1 -- 172.21.15.64:0/3492355256 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5f8054a90 con 0x7fd60015e1e0 2023-11-24T06:29:51.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.718+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fd60015e640 con 0x7fd5e006ae40 2023-11-24T06:29:51.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.718+0000 7fd5f6ffd640 1 -- 172.21.15.64:0/3492355256 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd5f0001c70 con 0x7fd5e006ae40 2023-11-24T06:29:51.721 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T06:29:51.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.720+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e006ae40 msgr2=0x7fd5e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.720+0000 7fd6048ad640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e006ae40 0x7fd5e006d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0000f60 tx=0x7fd5f0009040 comp rx=0 tx=0).stop 2023-11-24T06:29:51.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd60015e1e0 msgr2=0x7fd6000c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd60015e1e0 0x7fd6000c4390 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009300 tx=0x7fd5f800e490 comp rx=0 tx=0).stop 2023-11-24T06:29:51.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 shutdown_connections 2023-11-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fd5e006ae40 0x7fd5e006d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd60015e1e0 0x7fd6000c4390 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600143ae0 0x7fd6000c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 --2- 172.21.15.64:0/3492355256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000cf9a0 0x7fd6000cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 >> 172.21.15.64:0/3492355256 conn(0x7fd60006a800 msgr2=0x7fd60006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 shutdown_connections 2023-11-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.721+0000 7fd6048ad640 1 -- 172.21.15.64:0/3492355256 wait complete. 2023-11-24T06:29:51.743 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T06:29:51.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:51.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.970+0000 7fc50f59e640 1 Processor -- start 2023-11-24T06:29:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.971+0000 7fc50f59e640 1 -- start start 2023-11-24T06:29:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.972+0000 7fc50f59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc51013cfa0 0x7fc51015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.972+0000 7fc50f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 0x7fc5100b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.972+0000 7fc50f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5100cdec0 con 0x7fc5100cd530 2023-11-24T06:29:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.972+0000 7fc50f59e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5100ce1a0 con 0x7fc51013cfa0 2023-11-24T06:29:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.972+0000 7fc50f59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100ce480 con 0x7fc51013fcb0 2023-11-24T06:29:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.973+0000 7fc507fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc5100cd530 0x7fc51013f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35896/0 (socket says 172.21.15.64:35896) 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.973+0000 7fc507fff640 1 -- 172.21.15.64:0/2409074540 learned_addr learned my addr 172.21.15.64:0/2409074540 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.973+0000 7fc50ed9d640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 0x7fc5100b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.973+0000 7fc50e59c640 1 --2- 172.21.15.64:0/2409074540 >> v2:172.21.15.162:3300/0 conn(0x7fc51013cfa0 0x7fc51015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.973+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385369328 0 0) 0x7fc5100cdec0 con 0x7fc5100cd530 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.973+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4fc003520 con 0x7fc5100cd530 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2406376359 0 0) 0x7fc4fc003520 con 0x7fc5100cd530 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 msgr2=0x7fc5100b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 0x7fc5100b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 >> v2:172.21.15.162:3300/0 conn(0x7fc51013cfa0 msgr2=0x7fc51015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 --2- 172.21.15.64:0/2409074540 >> v2:172.21.15.162:3300/0 conn(0x7fc51013cfa0 0x7fc51015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100ce760 con 0x7fc5100cd530 2023-11-24T06:29:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7fc4f8001670 con 0x7fc5100cd530 2023-11-24T06:29:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 >> v1:172.21.15.64:6789/0 conn(0x7fc5100cd530 legacy=0x7fc51013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc003970 0x7fc4fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 0x7fc4fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc004d00 0x7fc4fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100ce1a0 con 0x7fc4fc003970 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100ce480 con 0x7fc4fc004d00 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc00dc10 con 0x7fc51013fcb0 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.974+0000 7fc50e59c640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc003970 0x7fc4fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc507fff640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 0x7fc4fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc50ed9d640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc004d00 0x7fc4fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc50e59c640 1 -- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 msgr2=0x7fc4fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc50e59c640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 0x7fc4fc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc50e59c640 1 -- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc004d00 msgr2=0x7fc4fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc50e59c640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc004d00 0x7fc4fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc50e59c640 1 -- 172.21.15.64:0/2409074540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc003520 con 0x7fc4fc003970 2023-11-24T06:29:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc507fff640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc51013fcb0 0x7fc4fc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.975+0000 7fc50ed9d640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc004d00 0x7fc4fc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.976+0000 7fc50e59c640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc003970 0x7fc4fc003d90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc500009ac0 tx=0x7fc4fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b193ab0be54ee35 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.976+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50002c710 con 0x7fc4fc003970 2023-11-24T06:29:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50002c870 con 0x7fc4fc003970 2023-11-24T06:29:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50dd9b640 1 -- 172.21.15.64:0/2409074540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50002cc00 con 0x7fc4fc003970 2023-11-24T06:29:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50f59e640 1 -- 172.21.15.64:0/2409074540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc003970 msgr2=0x7fc4fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50f59e640 1 --2- 172.21.15.64:0/2409074540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc003970 0x7fc4fc003d90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc500009ac0 tx=0x7fc4fc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50ed9d640 1 -- 172.21.15.64:0/2409074540 reap_dead start 2023-11-24T06:29:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50f59e640 1 -- 172.21.15.64:0/2409074540 shutdown_connections 2023-11-24T06:29:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50f59e640 1 -- 172.21.15.64:0/2409074540 >> 172.21.15.64:0/2409074540 conn(0x7fc51006a800 msgr2=0x7fc51006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50f59e640 1 -- 172.21.15.64:0/2409074540 shutdown_connections 2023-11-24T06:29:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.977+0000 7fc50f59e640 1 -- 172.21.15.64:0/2409074540 wait complete. 2023-11-24T06:29:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.979+0000 7fc50f59e640 1 Processor -- start 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.979+0000 7fc50f59e640 1 -- start start 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.980+0000 7fc50f59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc003970 0x7fc5101f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.980+0000 7fc50f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5101573c0 0x7fc5100af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.980+0000 7fc50f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc510149c80 0x7fc51014a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.980+0000 7fc50f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5101467e0 con 0x7fc510149c80 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.980+0000 7fc50f59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51014a640 con 0x7fc4fc003970 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.980+0000 7fc50f59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc510146a30 con 0x7fc5101573c0 2023-11-24T06:29:51.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc510149c80 0x7fc51014a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc510149c80 0x7fc51014a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46214/0 (socket says 172.21.15.64:46214) 2023-11-24T06:29:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 -- 172.21.15.64:0/1549915181 learned_addr learned my addr 172.21.15.64:0/1549915181 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc507fff640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5101573c0 0x7fc5100af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50e59c640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc003970 0x7fc5101f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 -- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5101573c0 msgr2=0x7fc5100af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5101573c0 0x7fc5100af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 -- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc003970 msgr2=0x7fc5101f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc003970 0x7fc5101f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50ed9d640 1 -- 172.21.15.64:0/1549915181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510146d10 con 0x7fc510149c80 2023-11-24T06:29:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.981+0000 7fc50e59c640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4fc003970 0x7fc5101f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.982+0000 7fc50ed9d640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc510149c80 0x7fc51014a100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc51006ce70 tx=0x7fc50800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.982+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508017400 con 0x7fc510149c80 2023-11-24T06:29:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.982+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc508011040 con 0x7fc510149c80 2023-11-24T06:29:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.982+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc510146ff0 con 0x7fc510149c80 2023-11-24T06:29:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.982+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508017780 con 0x7fc510149c80 2023-11-24T06:29:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.982+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51014aa40 con 0x7fc510149c80 2023-11-24T06:29:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.983+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc5100afb20 con 0x7fc510149c80 2023-11-24T06:29:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.984+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5080203b0 con 0x7fc510149c80 2023-11-24T06:29:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.985+0000 7fc506ffd640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc4f006ae70 0x7fc4f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.985+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc508089560 con 0x7fc510149c80 2023-11-24T06:29:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.985+0000 7fc50e59c640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc4f006ae70 0x7fc4f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.985+0000 7fc506ffd640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7fc4f0070cd0 0x7fc4f00730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.985+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 --> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4f00737a0 con 0x7fc4f0070cd0 2023-11-24T06:29:52.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.985+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc50808bc20 con 0x7fc510149c80 2023-11-24T06:29:52.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.986+0000 7fc50e59c640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc4f006ae70 0x7fc4f006d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc500036000 tx=0x7fc500030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.986+0000 7fc507fff640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7fc4f0070cd0 0x7fc4f00730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.987+0000 7fc507fff640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7fc4f0070cd0 0x7fc4f00730f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:51.989+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== osd.6 v2:172.21.15.162:6800/1044826159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc4f00737a0 con 0x7fc4f0070cd0 2023-11-24T06:29:52.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.004+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 --> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc51006d050 con 0x7fc4f0070cd0 2023-11-24T06:29:52.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.005+0000 7fc506ffd640 1 -- 172.21.15.64:0/1549915181 <== osd.6 v2:172.21.15.162:6800/1044826159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc4f800d040 con 0x7fc4f0070cd0 2023-11-24T06:29:52.008 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:52.008 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:52.008 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7fc4f0070cd0 msgr2=0x7fc4f00730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.162:6800/1044826159,v1:172.21.15.162:6801/1044826159] conn(0x7fc4f0070cd0 0x7fc4f00730f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc4f006ae70 msgr2=0x7fc4f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fc4f006ae70 0x7fc4f006d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc500036000 tx=0x7fc500030000 comp rx=0 tx=0).stop 2023-11-24T06:29:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc510149c80 msgr2=0x7fc51014a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 --2- 172.21.15.64:0/1549915181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc510149c80 0x7fc51014a100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc51006ce70 tx=0x7fc50800f4a0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50ed9d640 1 -- 172.21.15.64:0/1549915181 reap_dead start 2023-11-24T06:29:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 shutdown_connections 2023-11-24T06:29:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 >> 172.21.15.64:0/1549915181 conn(0x7fc51006a800 msgr2=0x7fc51006d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 shutdown_connections 2023-11-24T06:29:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.006+0000 7fc50f59e640 1 -- 172.21.15.64:0/1549915181 wait complete. 2023-11-24T06:29:52.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T06:29:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.245+0000 7f65ad239640 1 Processor -- start 2023-11-24T06:29:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.245+0000 7f65ad239640 1 -- start start 2023-11-24T06:29:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65ad239640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a8153c10 0x7f65a80b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65ad239640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f65a806d410 0x7f65a80ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65ad239640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a815aba0 con 0x7f65a80b3e40 2023-11-24T06:29:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65ad239640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a8159bf0 con 0x7f65a806d410 2023-11-24T06:29:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65ad239640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8159de0 con 0x7f65a8153c10 2023-11-24T06:29:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65aca38640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f65a806d410 0x7f65a80ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65aca38640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f65a806d410 0x7f65a80ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46216/0 (socket says 172.21.15.64:46216) 2023-11-24T06:29:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65a77fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f65a80b3e40 0x7f65a806cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35910/0 (socket says 172.21.15.64:35910) 2023-11-24T06:29:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.247+0000 7f65aca38640 1 -- 172.21.15.64:0/1758436495 learned_addr learned my addr 172.21.15.64:0/1758436495 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.248+0000 7f65a7fff640 1 --2- 172.21.15.64:0/1758436495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a8153c10 0x7f65a80b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.248+0000 7f65aca38640 1 -- 172.21.15.64:0/1758436495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a8153c10 msgr2=0x7f65a80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.248+0000 7f65aca38640 1 --2- 172.21.15.64:0/1758436495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a8153c10 0x7f65a80b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.248+0000 7f65aca38640 1 -- 172.21.15.64:0/1758436495 >> v1:172.21.15.64:6789/0 conn(0x7f65a80b3e40 legacy=0x7f65a806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.248+0000 7f65aca38640 1 -- 172.21.15.64:0/1758436495 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a815a0c0 con 0x7f65a806d410 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.248+0000 7f65a6ffd640 1 -- 172.21.15.64:0/1758436495 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136108739 0 0) 0x7f65a815aba0 con 0x7f65a80b3e40 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.249+0000 7f65aca38640 1 --2- 172.21.15.64:0/1758436495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a806d410 0x7f65a80ccd90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670897ca2e29f052 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.249+0000 7f65a6ffd640 1 -- 172.21.15.64:0/1758436495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0030840 con 0x7f65a806d410 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.249+0000 7f65a6ffd640 1 -- 172.21.15.64:0/1758436495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a00309a0 con 0x7f65a806d410 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.249+0000 7f65a6ffd640 1 -- 172.21.15.64:0/1758436495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0030d30 con 0x7f65a806d410 2023-11-24T06:29:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 -- 172.21.15.64:0/1758436495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a806d410 msgr2=0x7f65a80ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 --2- 172.21.15.64:0/1758436495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a806d410 0x7f65a80ccd90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002b980 comp rx=0 tx=0).stop 2023-11-24T06:29:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 -- 172.21.15.64:0/1758436495 shutdown_connections 2023-11-24T06:29:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 --2- 172.21.15.64:0/1758436495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a806d410 0x7f65a80ccd90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 --2- 172.21.15.64:0/1758436495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a8153c10 0x7f65a80b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 -- 172.21.15.64:0/1758436495 >> 172.21.15.64:0/1758436495 conn(0x7f65a806a3e0 msgr2=0x7f65a806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 -- 172.21.15.64:0/1758436495 shutdown_connections 2023-11-24T06:29:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.250+0000 7f65ad239640 1 -- 172.21.15.64:0/1758436495 wait complete. 2023-11-24T06:29:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.252+0000 7f65ad239640 1 Processor -- start 2023-11-24T06:29:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.252+0000 7f65ad239640 1 -- start start 2023-11-24T06:29:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.253+0000 7f65ad239640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65a806d410 0x7f65a81462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.253+0000 7f65ad239640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a80b3e40 0x7f65a814de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.253+0000 7f65ad239640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a8153c10 0x7f65a814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.253+0000 7f65ad239640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a815b660 con 0x7f65a8153c10 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.253+0000 7f65ad239640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a815ccb0 con 0x7f65a806d410 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.253+0000 7f65ad239640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a815ad50 con 0x7f65a80b3e40 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.253+0000 7f65aca38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a8153c10 0x7f65a814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a8153c10 0x7f65a814e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46222/0 (socket says 172.21.15.64:46222) 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 -- 172.21.15.64:0/3405808831 learned_addr learned my addr 172.21.15.64:0/3405808831 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65a77fe640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a80b3e40 0x7f65a814de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65a7fff640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65a806d410 0x7f65a81462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 -- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a80b3e40 msgr2=0x7f65a814de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a80b3e40 0x7f65a814de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 -- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65a806d410 msgr2=0x7f65a81462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65a806d410 0x7f65a81462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 -- 172.21.15.64:0/3405808831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00094d0 con 0x7f65a8153c10 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.254+0000 7f65aca38640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a8153c10 0x7f65a814e3b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f65a00090b0 tx=0x7f65a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.255+0000 7f65a4ff9640 1 -- 172.21.15.64:0/3405808831 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a003e040 con 0x7f65a8153c10 2023-11-24T06:29:52.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.255+0000 7f65a4ff9640 1 -- 172.21.15.64:0/3405808831 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0039c10 con 0x7f65a8153c10 2023-11-24T06:29:52.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.255+0000 7f65a4ff9640 1 -- 172.21.15.64:0/3405808831 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0004030 con 0x7f65a8153c10 2023-11-24T06:29:52.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.255+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a8149070 con 0x7f65a8153c10 2023-11-24T06:29:52.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.255+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a815afa0 con 0x7f65a8153c10 2023-11-24T06:29:52.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.256+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a815b7a0 con 0x7f65a8153c10 2023-11-24T06:29:52.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.257+0000 7f65a4ff9640 1 -- 172.21.15.64:0/3405808831 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65a0004190 con 0x7f65a8153c10 2023-11-24T06:29:52.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.258+0000 7f65a4ff9640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f658806ae70 0x7f658806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.258+0000 7f65a4ff9640 1 -- 172.21.15.64:0/3405808831 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65a00abd50 con 0x7f65a8153c10 2023-11-24T06:29:52.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.258+0000 7f65a7fff640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f658806ae70 0x7f658806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.259+0000 7f65a7fff640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f658806ae70 0x7f658806d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65980099c0 tx=0x7f6598003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.263+0000 7f65a4ff9640 1 -- 172.21.15.64:0/3405808831 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65a815b7a0 con 0x7f65a8153c10 2023-11-24T06:29:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.456+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f65a8153ff0 con 0x7f658806ae70 2023-11-24T06:29:52.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.457+0000 7f65a4ff9640 1 -- 172.21.15.64:0/3405808831 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f65980046c0 con 0x7f658806ae70 2023-11-24T06:29:52.460 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T06:29:52.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.459+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f658806ae70 msgr2=0x7f658806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.459+0000 7f65ad239640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f658806ae70 0x7f658806d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65980099c0 tx=0x7f6598003990 comp rx=0 tx=0).stop 2023-11-24T06:29:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.459+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a8153c10 msgr2=0x7f65a814e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.459+0000 7f65ad239640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a8153c10 0x7f65a814e3b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f65a00090b0 tx=0x7f65a0004730 comp rx=0 tx=0).stop 2023-11-24T06:29:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 shutdown_connections 2023-11-24T06:29:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f658806ae70 0x7f658806d330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65a8153c10 0x7f65a814e3b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65a80b3e40 0x7f65a814de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 --2- 172.21.15.64:0/3405808831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65a806d410 0x7f65a81462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 >> 172.21.15.64:0/3405808831 conn(0x7f65a806a3e0 msgr2=0x7f65a806cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:52.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 shutdown_connections 2023-11-24T06:29:52.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.460+0000 7f65ad239640 1 -- 172.21.15.64:0/3405808831 wait complete. 2023-11-24T06:29:52.482 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T06:29:52.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:29:52.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.708+0000 7fad5759e640 1 Processor -- start 2023-11-24T06:29:52.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.709+0000 7fad5759e640 1 -- start start 2023-11-24T06:29:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.710+0000 7fad5759e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad58071410 0x7fad58146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.710+0000 7fad5759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad58071f00 0x7fad58166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.710+0000 7fad5759e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad580b2e10 con 0x7fad5806c7c0 2023-11-24T06:29:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.710+0000 7fad5759e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5806cc50 con 0x7fad58071410 2023-11-24T06:29:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.710+0000 7fad5759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5806cec0 con 0x7fad58071f00 2023-11-24T06:29:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.711+0000 7fad5659c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad58071410 0x7fad58146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.711+0000 7fad56d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad58071f00 0x7fad58166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.711+0000 7fad5659c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad58071410 0x7fad58146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:37038/0 (socket says 172.21.15.64:37038) 2023-11-24T06:29:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.711+0000 7fad56d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad58071f00 0x7fad58166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.64:51866/0 (socket says 172.21.15.64:51866) 2023-11-24T06:29:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.711+0000 7fad5659c640 1 -- 172.21.15.64:0/1351433695 learned_addr learned my addr 172.21.15.64:0/1351433695 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.712+0000 7fad5659c640 1 -- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad58071f00 msgr2=0x7fad58166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:52.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.712+0000 7fad5659c640 1 --2- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad58071f00 0x7fad58166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.712+0000 7fad5659c640 1 -- 172.21.15.64:0/1351433695 >> v1:172.21.15.162:6789/0 conn(0x7fad5806c7c0 legacy=0x7fad580717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.712+0000 7fad5659c640 1 -- 172.21.15.64:0/1351433695 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5806d1a0 con 0x7fad58071410 2023-11-24T06:29:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.712+0000 7fad56d9d640 1 --2- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad58071f00 0x7fad58166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.714+0000 7fad5659c640 1 --2- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 0x7fad58146770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad48009870 tx=0x7fad4802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b42d62384e6413d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.714+0000 7fad55d9b640 1 -- 172.21.15.64:0/1351433695 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48030840 con 0x7fad58071410 2023-11-24T06:29:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.714+0000 7fad55d9b640 1 -- 172.21.15.64:0/1351433695 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480309a0 con 0x7fad58071410 2023-11-24T06:29:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.714+0000 7fad55d9b640 1 -- 172.21.15.64:0/1351433695 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48030d30 con 0x7fad58071410 2023-11-24T06:29:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 -- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 msgr2=0x7fad58146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 --2- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 0x7fad58146770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad48009870 tx=0x7fad4802b980 comp rx=0 tx=0).stop 2023-11-24T06:29:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 -- 172.21.15.64:0/1351433695 shutdown_connections 2023-11-24T06:29:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 --2- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad58071f00 0x7fad58166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 --2- 172.21.15.64:0/1351433695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 0x7fad58146770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 -- 172.21.15.64:0/1351433695 >> 172.21.15.64:0/1351433695 conn(0x7fad5806a800 msgr2=0x7fad5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 -- 172.21.15.64:0/1351433695 shutdown_connections 2023-11-24T06:29:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.715+0000 7fad5759e640 1 -- 172.21.15.64:0/1351433695 wait complete. 2023-11-24T06:29:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.717+0000 7fad5759e640 1 Processor -- start 2023-11-24T06:29:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.717+0000 7fad5759e640 1 -- start start 2023-11-24T06:29:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.718+0000 7fad5759e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad5806c7c0 0x7fad5815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.718+0000 7fad5759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 0x7fad581f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.718+0000 7fad5759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad58071f00 0x7fad581f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.718+0000 7fad5759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5813fc30 con 0x7fad58071f00 2023-11-24T06:29:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.718+0000 7fad5759e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5813f950 con 0x7fad58071410 2023-11-24T06:29:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.718+0000 7fad5759e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5815d8e0 con 0x7fad5806c7c0 2023-11-24T06:29:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.718+0000 7fad56d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad58071f00 0x7fad581f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad58071f00 0x7fad581f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46234/0 (socket says 172.21.15.64:46234) 2023-11-24T06:29:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 -- 172.21.15.64:0/2676756784 learned_addr learned my addr 172.21.15.64:0/2676756784 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad5659c640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad5806c7c0 0x7fad5815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad4ffff640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 0x7fad581f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 -- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad5806c7c0 msgr2=0x7fad5815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad5806c7c0 0x7fad5815cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 -- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 msgr2=0x7fad581f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad58071410 0x7fad581f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 -- 172.21.15.64:0/2676756784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad50009a70 con 0x7fad58071f00 2023-11-24T06:29:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.719+0000 7fad56d9d640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad58071f00 0x7fad581f6a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fad5000efc0 tx=0x7fad5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.720+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50009cd0 con 0x7fad58071f00 2023-11-24T06:29:52.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.720+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5000f040 con 0x7fad58071f00 2023-11-24T06:29:52.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.720+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50003990 con 0x7fad58071f00 2023-11-24T06:29:52.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.720+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad480094d0 con 0x7fad58071f00 2023-11-24T06:29:52.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.720+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5815dea0 con 0x7fad58071f00 2023-11-24T06:29:52.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.721+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad5806da40 con 0x7fad58071f00 2023-11-24T06:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.722+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad50014480 con 0x7fad58071f00 2023-11-24T06:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.723+0000 7fad4effd640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fad3806ae40 0x7fad3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.723+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad50089ef0 con 0x7fad58071f00 2023-11-24T06:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.723+0000 7fad5659c640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fad3806ae40 0x7fad3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.724+0000 7fad4effd640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fad38070ca0 0x7fad380730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.724+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 --> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad38073770 con 0x7fad38070ca0 2023-11-24T06:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.724+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad5008c5b0 con 0x7fad58071f00 2023-11-24T06:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.724+0000 7fad5659c640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fad3806ae40 0x7fad3806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fad48036000 tx=0x7fad48008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.724+0000 7fad4ffff640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fad38070ca0 0x7fad380730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.725+0000 7fad4ffff640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fad38070ca0 0x7fad380730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.727+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== osd.7 v2:172.21.15.162:6824/4255674423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fad38073770 con 0x7fad38070ca0 2023-11-24T06:29:52.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.743+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 --> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fad5806d760 con 0x7fad38070ca0 2023-11-24T06:29:52.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.744+0000 7fad4effd640 1 -- 172.21.15.64:0/2676756784 <== osd.7 v2:172.21.15.162:6824/4255674423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fad400050b0 con 0x7fad38070ca0 2023-11-24T06:29:52.747 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T06:29:52.747 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:29:52.747 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T06:29:52.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fad38070ca0 msgr2=0x7fad380730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.162:6824/4255674423,v1:172.21.15.162:6825/4255674423] conn(0x7fad38070ca0 0x7fad380730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fad3806ae40 msgr2=0x7fad3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7fad3806ae40 0x7fad3806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fad48036000 tx=0x7fad48008030 comp rx=0 tx=0).stop 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad58071f00 msgr2=0x7fad581f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 --2- 172.21.15.64:0/2676756784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad58071f00 0x7fad581f6a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fad5000efc0 tx=0x7fad5000c490 comp rx=0 tx=0).stop 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad56d9d640 1 -- 172.21.15.64:0/2676756784 reap_dead start 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 shutdown_connections 2023-11-24T06:29:52.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 >> 172.21.15.64:0/2676756784 conn(0x7fad5806a800 msgr2=0x7fad580b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:52.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 shutdown_connections 2023-11-24T06:29:52.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.745+0000 7fad5759e640 1 -- 172.21.15.64:0/2676756784 wait complete. 2023-11-24T06:29:52.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T06:29:52.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.990+0000 7f79b5901640 1 Processor -- start 2023-11-24T06:29:52.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.991+0000 7f79b5901640 1 -- start start 2023-11-24T06:29:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79b5901640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79b013e360 0x7f79b00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79b5901640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b015c5f0 con 0x7f79b015f9c0 2023-11-24T06:29:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79b5901640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b006c470 con 0x7f79b013df80 2023-11-24T06:29:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79b5901640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b006c750 con 0x7f79b013e360 2023-11-24T06:29:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79a7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79b013e360 0x7f79b00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79a7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79b013e360 0x7f79b00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33500/0 (socket says 172.21.15.64:33500) 2023-11-24T06:29:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79a7fff640 1 -- 172.21.15.64:0/2278975797 learned_addr learned my addr 172.21.15.64:0/2278975797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.993+0000 7f79af7fe640 1 --1- 172.21.15.64:0/2278975797 >> v1:172.21.15.64:6789/0 conn(0x7f79b015f9c0 0x7f79b00cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49062/0 (socket says 172.21.15.64:49062) 2023-11-24T06:29:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.994+0000 7f79a7fff640 1 -- 172.21.15.64:0/2278975797 >> v1:172.21.15.162:6789/0 conn(0x7f79b013df80 legacy=0x7f79b015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.994+0000 7f79a7fff640 1 -- 172.21.15.64:0/2278975797 >> v1:172.21.15.64:6789/0 conn(0x7f79b015f9c0 legacy=0x7f79b00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.994+0000 7f79a7fff640 1 -- 172.21.15.64:0/2278975797 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b006ca30 con 0x7f79b013e360 2023-11-24T06:29:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.995+0000 7f79a7fff640 1 --2- 172.21.15.64:0/2278975797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013e360 0x7f79b00cf290 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7998009870 tx=0x7f799802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58809ddd65f75be8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.995+0000 7f79ae7fc640 1 -- 172.21.15.64:0/2278975797 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998030860 con 0x7f79b013e360 2023-11-24T06:29:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.995+0000 7f79ae7fc640 1 -- 172.21.15.64:0/2278975797 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79980309c0 con 0x7f79b013e360 2023-11-24T06:29:52.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.995+0000 7f79ae7fc640 1 -- 172.21.15.64:0/2278975797 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998030d50 con 0x7f79b013e360 2023-11-24T06:29:53.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.996+0000 7f79b5901640 1 -- 172.21.15.64:0/2278975797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013e360 msgr2=0x7f79b00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.996+0000 7f79b5901640 1 --2- 172.21.15.64:0/2278975797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013e360 0x7f79b00cf290 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7998009870 tx=0x7f799802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.996+0000 7f79b5901640 1 -- 172.21.15.64:0/2278975797 shutdown_connections 2023-11-24T06:29:53.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.996+0000 7f79b5901640 1 --2- 172.21.15.64:0/2278975797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013e360 0x7f79b00cf290 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.996+0000 7f79b5901640 1 -- 172.21.15.64:0/2278975797 >> 172.21.15.64:0/2278975797 conn(0x7f79b006a800 msgr2=0x7f79b006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.996+0000 7f79b5901640 1 -- 172.21.15.64:0/2278975797 shutdown_connections 2023-11-24T06:29:53.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.996+0000 7f79b5901640 1 -- 172.21.15.64:0/2278975797 wait complete. 2023-11-24T06:29:53.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.998+0000 7f79b5901640 1 Processor -- start 2023-11-24T06:29:53.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.998+0000 7f79b5901640 1 -- start start 2023-11-24T06:29:53.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79b5901640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013df80 0x7f79b00a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79b5901640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79b013e360 0x7f79b01532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79b5901640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79b015f9c0 0x7f79b01f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79b5901640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0158d30 con 0x7f79b013df80 2023-11-24T06:29:53.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79b5901640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01537e0 con 0x7f79b015f9c0 2023-11-24T06:29:53.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79b5901640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01f1800 con 0x7f79b013e360 2023-11-24T06:29:53.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013df80 0x7f79b00a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013df80 0x7f79b00a85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33508/0 (socket says 172.21.15.64:33508) 2023-11-24T06:29:53.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:52.999+0000 7f79aeffd640 1 -- 172.21.15.64:0/70411290 learned_addr learned my addr 172.21.15.64:0/70411290 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:53.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79a7fff640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79b013e360 0x7f79b01532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79af7fe640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79b015f9c0 0x7f79b01f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79aeffd640 1 -- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79b013e360 msgr2=0x7f79b01532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79aeffd640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79b013e360 0x7f79b01532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79aeffd640 1 -- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79b015f9c0 msgr2=0x7f79b01f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79aeffd640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79b015f9c0 0x7f79b01f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79aeffd640 1 -- 172.21.15.64:0/70411290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79980094d0 con 0x7f79b013df80 2023-11-24T06:29:53.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79af7fe640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79b015f9c0 0x7f79b01f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:53.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79aeffd640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013df80 0x7f79b00a85e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f79a000b420 tx=0x7f79a000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.000+0000 7f79acff9640 1 -- 172.21.15.64:0/70411290 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0011830 con 0x7f79b013df80 2023-11-24T06:29:53.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.001+0000 7f79acff9640 1 -- 172.21.15.64:0/70411290 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0011990 con 0x7f79b013df80 2023-11-24T06:29:53.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.001+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b01f1ae0 con 0x7f79b013df80 2023-11-24T06:29:53.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.001+0000 7f79acff9640 1 -- 172.21.15.64:0/70411290 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0011d40 con 0x7f79b013df80 2023-11-24T06:29:53.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.001+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0158f40 con 0x7f79b013df80 2023-11-24T06:29:53.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.002+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b00675d0 con 0x7f79b013df80 2023-11-24T06:29:53.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.004+0000 7f79acff9640 1 -- 172.21.15.64:0/70411290 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79a001b460 con 0x7f79b013df80 2023-11-24T06:29:53.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.005+0000 7f79acff9640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.005+0000 7f79acff9640 1 -- 172.21.15.64:0/70411290 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f79a0089100 con 0x7f79b013df80 2023-11-24T06:29:53.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.005+0000 7f79a7fff640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.006+0000 7f79a7fff640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f799006ae40 0x7f799006d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7998036000 tx=0x7f7998008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.009+0000 7f79acff9640 1 -- 172.21.15.64:0/70411290 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79a0054410 con 0x7f79b013df80 2023-11-24T06:29:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.208+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f79b006cff0 con 0x7f799006ae40 2023-11-24T06:29:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.209+0000 7f79acff9640 1 -- 172.21.15.64:0/70411290 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7998044470 con 0x7f799006ae40 2023-11-24T06:29:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T06:29:53.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f799006ae40 msgr2=0x7f799006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f799006ae40 0x7f799006d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7998036000 tx=0x7f7998008030 comp rx=0 tx=0).stop 2023-11-24T06:29:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013df80 msgr2=0x7f79b00a85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013df80 0x7f79b00a85e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f79a000b420 tx=0x7f79a000b8e0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 shutdown_connections 2023-11-24T06:29:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79b015f9c0 0x7f79b01f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79b013e360 0x7f79b01532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.211+0000 7f79b5901640 1 --2- 172.21.15.64:0/70411290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b013df80 0x7f79b00a85e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.212+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 >> 172.21.15.64:0/70411290 conn(0x7f79b006a800 msgr2=0x7f79b006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.212+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 shutdown_connections 2023-11-24T06:29:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.212+0000 7f79b5901640 1 -- 172.21.15.64:0/70411290 wait complete. 2023-11-24T06:29:53.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:53.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.458+0000 7f2927413640 1 Processor -- start 2023-11-24T06:29:53.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.458+0000 7f2927413640 1 -- start start 2023-11-24T06:29:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.460+0000 7f2927413640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 0x7f29200cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.460+0000 7f2927413640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2920070e50 con 0x7f292006d160 2023-11-24T06:29:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.460+0000 7f2927413640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200ccea0 con 0x7f2920143c40 2023-11-24T06:29:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.460+0000 7f2927413640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200723c0 con 0x7f29200cc500 2023-11-24T06:29:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f2926c12640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f292006d160 0x7f292006d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49064/0 (socket says 172.21.15.64:49064) 2023-11-24T06:29:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f2926c12640 1 -- 172.21.15.64:0/206957139 learned_addr learned my addr 172.21.15.64:0/206957139 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:29:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f2925c10640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 0x7f29200cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837395047 0 0) 0x7f2920070e50 con 0x7f292006d160 2023-11-24T06:29:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f290c003520 con 0x7f292006d160 2023-11-24T06:29:53.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3420867558 0 0) 0x7f290c003520 con 0x7f292006d160 2023-11-24T06:29:53.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 msgr2=0x7f29200cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:53.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.461+0000 7f292540f640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 0x7f29200cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 >> v1:172.21.15.162:6789/0 conn(0x7f2920143c40 legacy=0x7f2920071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920144020 con 0x7f292006d160 2023-11-24T06:29:53.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f29180019f0 con 0x7f292006d160 2023-11-24T06:29:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 >> v1:172.21.15.64:6789/0 conn(0x7f292006d160 legacy=0x7f292006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 0x7f290c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f290c003f70 0x7f290c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c004d00 0x7f290c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200723c0 con 0x7f290c003f70 2023-11-24T06:29:53.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c00bb70 con 0x7f290c004d00 2023-11-24T06:29:53.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c00bd30 con 0x7f29200cc500 2023-11-24T06:29:53.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f2925c10640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f290c003f70 0x7f290c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f2926411640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 0x7f290c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f2926c12640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c004d00 0x7f290c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.462+0000 7f2925c10640 1 -- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 msgr2=0x7f290c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.463+0000 7f2925c10640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200cc500 0x7f290c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.463+0000 7f2925c10640 1 -- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c004d00 msgr2=0x7f290c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:53.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.463+0000 7f2925c10640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c004d00 0x7f290c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.463+0000 7f2925c10640 1 -- 172.21.15.64:0/206957139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c003520 con 0x7f290c003f70 2023-11-24T06:29:53.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.464+0000 7f2925c10640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f290c003f70 0x7f290c0047c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2914009ac0 tx=0x7f290c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bc89a06aee4c8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.464+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291402c6b0 con 0x7f290c003f70 2023-11-24T06:29:53.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.464+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291402c810 con 0x7f290c003f70 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.464+0000 7f292540f640 1 -- 172.21.15.64:0/206957139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291402cba0 con 0x7f290c003f70 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.465+0000 7f2927413640 1 -- 172.21.15.64:0/206957139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f290c003f70 msgr2=0x7f290c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.465+0000 7f2927413640 1 --2- 172.21.15.64:0/206957139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f290c003f70 0x7f290c0047c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2914009ac0 tx=0x7f290c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.465+0000 7f2926c12640 1 -- 172.21.15.64:0/206957139 reap_dead start 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.465+0000 7f2927413640 1 -- 172.21.15.64:0/206957139 shutdown_connections 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.465+0000 7f2927413640 1 -- 172.21.15.64:0/206957139 >> 172.21.15.64:0/206957139 conn(0x7f292006a800 msgr2=0x7f292006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.465+0000 7f2927413640 1 -- 172.21.15.64:0/206957139 shutdown_connections 2023-11-24T06:29:53.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.465+0000 7f2927413640 1 -- 172.21.15.64:0/206957139 wait complete. 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.467+0000 7f2927413640 1 Processor -- start 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.467+0000 7f2927413640 1 -- start start 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2927413640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c003f70 0x7f2920143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2927413640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2920159a70 0x7f29201e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2927413640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200af1f0 0x7f29200af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2927413640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920142cf0 con 0x7f2920159a70 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2927413640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200afbc0 con 0x7f290c003f70 2023-11-24T06:29:53.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2927413640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29201e1f10 con 0x7f29200af1f0 2023-11-24T06:29:53.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2925c10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2920159a70 0x7f29201e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2925c10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2920159a70 0x7f29201e19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33522/0 (socket says 172.21.15.64:33522) 2023-11-24T06:29:53.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.468+0000 7f2925c10640 1 -- 172.21.15.64:0/297649950 learned_addr learned my addr 172.21.15.64:0/297649950 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:29:53.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2925c10640 1 -- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200af1f0 msgr2=0x7f29200af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2926c12640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200af1f0 0x7f29200af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2926411640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c003f70 0x7f2920143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2925c10640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200af1f0 0x7f29200af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2925c10640 1 -- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c003f70 msgr2=0x7f2920143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2925c10640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c003f70 0x7f2920143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2925c10640 1 -- 172.21.15.64:0/297649950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920159ef0 con 0x7f2920159a70 2023-11-24T06:29:53.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2925c10640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2920159a70 0x7f29201e19d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2914037040 tx=0x7f2914004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2912ffd640 1 -- 172.21.15.64:0/297649950 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914003810 con 0x7f2920159a70 2023-11-24T06:29:53.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2912ffd640 1 -- 172.21.15.64:0/297649950 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2914003970 con 0x7f2920159a70 2023-11-24T06:29:53.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2912ffd640 1 -- 172.21.15.64:0/297649950 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914003ce0 con 0x7f2920159a70 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.469+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2920142230 con 0x7f2920159a70 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.470+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2920142630 con 0x7f2920159a70 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.471+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920142480 con 0x7f2920159a70 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.472+0000 7f2912ffd640 1 -- 172.21.15.64:0/297649950 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2914047450 con 0x7f2920159a70 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.473+0000 7f2912ffd640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.473+0000 7f2912ffd640 1 -- 172.21.15.64:0/297649950 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f29140ab610 con 0x7f2920159a70 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.473+0000 7f2926411640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.474+0000 7f2926411640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f290406ae40 0x7f290406d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f291c007fd0 tx=0x7f291c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.478+0000 7f2912ffd640 1 -- 172.21.15.64:0/297649950 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2914076a60 con 0x7f2920159a70 2023-11-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.666+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f292006c960 con 0x7f290406ae40 2023-11-24T06:29:53.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.672+0000 7f2912ffd640 1 -- 172.21.15.64:0/297649950 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48433 (secure 0 0 0) 0x7f291c00d050 con 0x7f290406ae40 2023-11-24T06:29:53.676 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:29:53.676 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:29:53.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f290406ae40 msgr2=0x7f290406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f290406ae40 0x7f290406d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f291c007fd0 tx=0x7f291c009040 comp rx=0 tx=0).stop 2023-11-24T06:29:53.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2920159a70 msgr2=0x7f29201e19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2920159a70 0x7f29201e19d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2914037040 tx=0x7f2914004730 comp rx=0 tx=0).stop 2023-11-24T06:29:53.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 shutdown_connections 2023-11-24T06:29:53.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29200af1f0 0x7f29200af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2920159a70 0x7f29201e19d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 --2- 172.21.15.64:0/297649950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f290c003f70 0x7f2920143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 >> 172.21.15.64:0/297649950 conn(0x7f292006a800 msgr2=0x7f292006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 shutdown_connections 2023-11-24T06:29:53.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:29:53.675+0000 7f2927413640 1 -- 172.21.15.64:0/297649950 wait complete. 2023-11-24T06:29:53.699 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2023-11-24T06:29:53.140255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220856,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753864,"statfs":{"total":773094113280,"available":772867956736,"internally_reserved":0,"allocated":4792320,"data_stored":2599216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003066"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-24T06:29:39.128458+0000","last_change":"2023-11-24T06:27:36.913986+0000","last_active":"2023-11-24T06:29:39.128458+0000","last_peered":"2023-11-24T06:29:39.128458+0000","last_clean":"2023-11-24T06:29:39.128458+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:29:39.128458+0000","last_undegraded":"2023-11-24T06:29:39.128458+0000","last_fullsized":"2023-11-24T06:29:39.128458+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:35.857721+0000","last_clean_scrub_stamp":"2023-11-24T06:27:35.857721+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:35:11.841096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97399999999999998,"15min":0.97699999999999998},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.633,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0469999999999999,"15min":1.0589999999999999},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7669999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.22}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0529999999999999,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.141,"15min":1.169},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.581,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.3799999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0960000000000001,"15min":1.091},"min":{"1min":0.64100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.1779999999999999,"15min":1.1859999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.145,"15min":1.149},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.474},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.347,"15min":1.369},"min":{"1min":0.65300000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.871,"5min":2.024,"15min":2.024},"last":1.5149999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.758,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.117},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3380000000000001,"15min":1.343},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.6080000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.379,"15min":1.383},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.093,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.56599999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.853,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.5509999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.476,"15min":1.468},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.655},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.397,"15min":1.393},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.7110000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.526,"5min":2.323,"15min":2.323},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97299999999999998,"15min":0.96999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.86299999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.0700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5669999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.01},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.048,"15min":1.0329999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.75},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.131,"15min":1.1259999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.236}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.151,"15min":1.167},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2290000000000001,"15min":1.2470000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.3540000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.288,"15min":1.302},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.397},{"interface":"front","average":{"1min":1.3,"5min":1.379,"15min":1.3919999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.847,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.071}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.425,"15min":1.446},"min":{"1min":0.77900000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.887,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.294},{"interface":"front","average":{"1min":1.296,"5min":1.417,"15min":1.4370000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8440000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.98099999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5289999999999999,"15min":1.5549999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.839,"5min":3.052,"15min":3.052},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.732,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.4590000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.53100000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.399},{"interface":"front","average":{"1min":1.016,"5min":1.1120000000000001,"15min":1.1279999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.671,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.278}]},{"osd":1,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.105,"15min":1.1140000000000001},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.6439999999999999,"5min":1.79,"15min":1.79},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.224,"15min":1.24},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8320000000000001,"5min":1.857,"15min":1.857},"last":1.615}]},{"osd":2,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.222,"15min":1.24},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.897,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.153},{"interface":"front","average":{"1min":1.107,"5min":1.2589999999999999,"15min":1.2849999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6499999999999999,"5min":1.986,"15min":1.986},"last":1.4830000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.849,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.329,"15min":1.361},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.746,"5min":2.359,"15min":2.359},"last":1.4430000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.3260000000000001,"15min":1.3580000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.702,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.377,"15min":1.385},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1160000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.528}]},{"osd":6,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.4299999999999999,"15min":1.46},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9630000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.464,"15min":1.488},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9610000000000001,"5min":3.0299999999999998,"15min":3.0299999999999998},"last":1.6819999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5249999999999999,"15min":1.5389999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2570000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.72},{"interface":"front","average":{"1min":1.381,"5min":1.5880000000000001,"15min":1.6220000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1970000000000001,"5min":2.552,"15min":2.552},"last":1.647}]}]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.113,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2210000000000001,"15min":1.2370000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.83499999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64600000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.8879999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1459999999999999,"15min":1.1479999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.978,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.4810000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6870000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.244},"min":{"1min":0.69999999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0840000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.371},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2480000000000002,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.2130000000000001,"15min":1.23},"min":{"1min":0.58399999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0659999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.454,"15min":1.474},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2400000000000002,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.51,"15min":1.53},"min":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.157,"5min":4.2060000000000004,"15min":4.2060000000000004},"last":1.583}]},{"osd":6,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.381,"15min":1.401},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8220000000000001,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5580000000000001,"15min":1.577},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.3029999999999999,"5min":4.335,"15min":4.335},"last":1.6240000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":3.7959999999999998,"5min":2.0779999999999998,"15min":1.792},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":40.311999999999998,"5min":40.311999999999998,"15min":40.311999999999998},"last":1.671},{"interface":"front","average":{"1min":3.7799999999999998,"5min":2.0840000000000001,"15min":1.802},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":40.286000000000001,"5min":40.286000000000001,"15min":40.286000000000001},"last":1.5549999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.443},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.83799999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.288,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.99299999999999999,"15min":1.014},"min":{"1min":0.501,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4530000000000001,"5min":1.629,"15min":1.629},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.107,"15min":1.143},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.401,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2210000000000001,"15min":1.2250000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.86599999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5960000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2829999999999999,"15min":1.288},"min":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.8500000000000001,"5min":1.885,"15min":1.885},"last":1.1459999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":0.68200000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7849999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.349,"15min":1.3700000000000001},"min":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9550000000000001,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.204}]},{"osd":7,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.421,"15min":1.4410000000000001},"min":{"1min":0.76800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9079999999999999,"15min":1.613},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":40.671999999999997,"5min":40.671999999999997,"15min":40.671999999999997},"last":1.2430000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91600000000000004,"15min":0.93600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.5469999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.024},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3169999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.044,"15min":1.0780000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.569,"5min":2.278,"15min":2.278},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.353,"5min":1.847,"15min":1.847},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.232,"15min":1.2649999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7130000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.323,"15min":1.345},"min":{"1min":0.69099999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7450000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.2090000000000001,"15min":1.2350000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.712,"5min":3.3839999999999999,"15min":3.3839999999999999},"last":0.91600000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.345,"15min":1.361},"min":{"1min":0.87,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.821,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.115},{"interface":"front","average":{"1min":1.153,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8480000000000001,"5min":3.706,"15min":3.706},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2869999999999999,"15min":1.319},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7729999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4019999999999999,"15min":1.421},"min":{"1min":0.872,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.913,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.46,"15min":1.5029999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.99,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.4119999999999999,"15min":1.4590000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.9910000000000001,"5min":3.661,"15min":3.661},"last":1.153}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96499999999999997,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3160000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.016,"15min":1.026},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94499999999999995,"15min":0.96199999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.034,"15min":1.052},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.534,"5min":1.998,"15min":1.998},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.1140000000000001,"15min":1.137},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7589999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.028,"5min":1.079,"15min":1.087},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6870000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.226,"15min":1.26},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4650000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.012},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3720000000000001,"15min":1.3759999999999999},"min":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.905,"5min":2.415,"15min":2.415},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.167,"5min":1.365,"15min":1.3979999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.8360000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.47,"15min":1.48},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1259999999999999,"5min":2.544,"15min":2.544},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.4630000000000001,"15min":1.4890000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.909,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.081}]},{"osd":7,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.488,"15min":1.504},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0270000000000001,"5min":2.355,"15min":2.355},"last":1.337},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5329999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.056,"5min":2.492,"15min":2.492},"last":0.93899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.94699999999999995,"15min":0.98699999999999999},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.133,"5min":2.54,"15min":2.54},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.95199999999999996,"15min":0.98799999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.599,"5min":1.607,"15min":1.607},"last":1.0589999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.0840000000000001,"15min":1.1319999999999999},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.2609999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.177},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.91200000000000003,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.107,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.123},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4410000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.173,"15min":1.206},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5700000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.2569999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3660000000000001,"15min":1.403},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5209999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.494,"15min":1.542},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5209999999999999,"5min":2.456,"15min":2.456},"last":1.284}]},{"osd":5,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3999999999999999,"15min":1.427},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.4390000000000001,"15min":1.5069999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.587,"15min":1.665},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7470000000000001,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.258,"5min":1.5429999999999999,"15min":1.591},"min":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.8879999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.393}]},{"osd":7,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.4510000000000001,"15min":1.5089999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7929999999999999,"5min":2.802,"15min":2.802},"last":1.024},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.631,"15min":1.698},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.8859999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.139}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:53.702 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T06:27:35.857721+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=27, tm_sec=35, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=29, tm_sec=46, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T06:29:53.702 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T06:29:59.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:29:59.286 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:13.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:30:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.937+0000 7f40f22ca640 1 Processor -- start 2023-11-24T06:30:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.937+0000 7f40f22ca640 1 -- start start 2023-11-24T06:30:13.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.939+0000 7f40f22ca640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40ec071410 0x7f40ec146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.939+0000 7f40f22ca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec071f00 0x7f40ec166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.939+0000 7f40f22ca640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec0b2e10 con 0x7f40ec06c7c0 2023-11-24T06:30:13.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.939+0000 7f40f22ca640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec06cc50 con 0x7f40ec071410 2023-11-24T06:30:13.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.939+0000 7f40f22ca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec06cec0 con 0x7f40ec071f00 2023-11-24T06:30:13.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.940+0000 7f40f12c8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40ec071410 0x7f40ec146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.940+0000 7f40f1ac9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec071f00 0x7f40ec166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.940+0000 7f40f12c8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40ec071410 0x7f40ec146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.64:59724/0 (socket says 172.21.15.64:59724) 2023-11-24T06:30:13.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.940+0000 7f40f12c8640 1 -- 172.21.15.64:0/481456947 learned_addr learned my addr 172.21.15.64:0/481456947 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:30:13.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.940+0000 7f40f0ac7640 1 --1- 172.21.15.64:0/481456947 >> v1:172.21.15.162:6789/0 conn(0x7f40ec06c7c0 0x7f40ec0717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.64:34160/0 (socket says 172.21.15.64:34160) 2023-11-24T06:30:13.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.941+0000 7f40f12c8640 1 -- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec071f00 msgr2=0x7f40ec166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:13.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.941+0000 7f40f12c8640 1 --2- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec071f00 0x7f40ec166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.941+0000 7f40f12c8640 1 -- 172.21.15.64:0/481456947 >> v1:172.21.15.162:6789/0 conn(0x7f40ec06c7c0 legacy=0x7f40ec0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.941+0000 7f40f12c8640 1 -- 172.21.15.64:0/481456947 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec06d1a0 con 0x7f40ec071410 2023-11-24T06:30:13.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.941+0000 7f40f1ac9640 1 --2- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec071f00 0x7f40ec166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:30:13.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.941+0000 7f40e3fff640 1 -- 172.21.15.64:0/481456947 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981864695 0 0) 0x7f40ec0b2e10 con 0x7f40ec06c7c0 2023-11-24T06:30:13.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.943+0000 7f40f12c8640 1 --2- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071410 0x7f40ec146770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=928c3c429f61d496 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.943+0000 7f40e3fff640 1 -- 172.21.15.64:0/481456947 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8030840 con 0x7f40ec071410 2023-11-24T06:30:13.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40e3fff640 1 -- 172.21.15.64:0/481456947 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e80309a0 con 0x7f40ec071410 2023-11-24T06:30:13.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40e3fff640 1 -- 172.21.15.64:0/481456947 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8030d30 con 0x7f40ec071410 2023-11-24T06:30:13.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40f22ca640 1 -- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071410 msgr2=0x7f40ec146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40f22ca640 1 --2- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071410 0x7f40ec146770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e802b980 comp rx=0 tx=0).stop 2023-11-24T06:30:13.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40f22ca640 1 -- 172.21.15.64:0/481456947 shutdown_connections 2023-11-24T06:30:13.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40f22ca640 1 --2- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec071f00 0x7f40ec166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40f22ca640 1 --2- 172.21.15.64:0/481456947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071410 0x7f40ec146770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.944+0000 7f40f22ca640 1 -- 172.21.15.64:0/481456947 >> 172.21.15.64:0/481456947 conn(0x7f40ec06a800 msgr2=0x7f40ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:13.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.945+0000 7f40f22ca640 1 -- 172.21.15.64:0/481456947 shutdown_connections 2023-11-24T06:30:13.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.945+0000 7f40f22ca640 1 -- 172.21.15.64:0/481456947 wait complete. 2023-11-24T06:30:13.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.947+0000 7f40f22ca640 1 Processor -- start 2023-11-24T06:30:13.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.947+0000 7f40f22ca640 1 -- start start 2023-11-24T06:30:13.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.948+0000 7f40f22ca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec06c7c0 0x7f40ec139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.948+0000 7f40f22ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ec071410 0x7f40ec15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.948+0000 7f40f22ca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071f00 0x7f40ec13cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.948+0000 7f40f22ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec15e580 con 0x7f40ec071410 2023-11-24T06:30:13.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.948+0000 7f40f22ca640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec15fa90 con 0x7f40ec071f00 2023-11-24T06:30:13.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.948+0000 7f40f22ca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec13a270 con 0x7f40ec06c7c0 2023-11-24T06:30:13.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ec071410 0x7f40ec15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ec071410 0x7f40ec15c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39414/0 (socket says 172.21.15.64:39414) 2023-11-24T06:30:13.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 -- 172.21.15.64:0/2206196073 learned_addr learned my addr 172.21.15.64:0/2206196073 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:30:13.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f1ac9640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071f00 0x7f40ec13cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f12c8640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec06c7c0 0x7f40ec139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 -- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec06c7c0 msgr2=0x7f40ec139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec06c7c0 0x7f40ec139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 -- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071f00 msgr2=0x7f40ec13cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071f00 0x7f40ec13cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.949+0000 7f40f0ac7640 1 -- 172.21.15.64:0/2206196073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e4009a70 con 0x7f40ec071410 2023-11-24T06:30:13.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.950+0000 7f40f0ac7640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ec071410 0x7f40ec15c9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00c4d0 tx=0x7f40dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.950+0000 7f40e1ffb640 1 -- 172.21.15.64:0/2206196073 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0074e0 con 0x7f40ec071410 2023-11-24T06:30:13.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.950+0000 7f40e1ffb640 1 -- 172.21.15.64:0/2206196073 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc007640 con 0x7f40ec071410 2023-11-24T06:30:13.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.950+0000 7f40e1ffb640 1 -- 172.21.15.64:0/2206196073 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc004a20 con 0x7f40ec071410 2023-11-24T06:30:13.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.950+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e80094d0 con 0x7f40ec071410 2023-11-24T06:30:13.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.950+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec1f6750 con 0x7f40ec071410 2023-11-24T06:30:13.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.952+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec15e6f0 con 0x7f40ec071410 2023-11-24T06:30:13.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.953+0000 7f40e1ffb640 1 -- 172.21.15.64:0/2206196073 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40dc004b80 con 0x7f40ec071410 2023-11-24T06:30:13.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.953+0000 7f40e1ffb640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f40d006ae70 0x7f40d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.954+0000 7f40e1ffb640 1 -- 172.21.15.64:0/2206196073 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f40dc0890d0 con 0x7f40ec071410 2023-11-24T06:30:13.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.954+0000 7f40f12c8640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f40d006ae70 0x7f40d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.955+0000 7f40f12c8640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f40d006ae70 0x7f40d006d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f40e8036000 tx=0x7f40e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:13.959+0000 7f40e1ffb640 1 -- 172.21.15.64:0/2206196073 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40dc054480 con 0x7f40ec071410 2023-11-24T06:30:14.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.148+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 --> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40ec06da40 con 0x7f40d006ae70 2023-11-24T06:30:14.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.151+0000 7f40e1ffb640 1 -- 172.21.15.64:0/2206196073 <== mgr.4107 v2:172.21.15.64:6832/1000290349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48278 (secure 0 0 0) 0x7f40e8044470 con 0x7f40d006ae70 2023-11-24T06:30:14.155 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T06:30:14.156 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T06:30:14.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f40d006ae70 msgr2=0x7f40d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f40d006ae70 0x7f40d006d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f40e8036000 tx=0x7f40e8008030 comp rx=0 tx=0).stop 2023-11-24T06:30:14.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ec071410 msgr2=0x7f40ec15c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ec071410 0x7f40ec15c9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00c4d0 tx=0x7f40dc00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 shutdown_connections 2023-11-24T06:30:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f40d006ae70 0x7f40d006d330 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40ec071f00 0x7f40ec13cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40ec071410 0x7f40ec15c9d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 --2- 172.21.15.64:0/2206196073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40ec06c7c0 0x7f40ec139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.154+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 >> 172.21.15.64:0/2206196073 conn(0x7f40ec06a800 msgr2=0x7f40ec0b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.155+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 shutdown_connections 2023-11-24T06:30:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.155+0000 7f40f22ca640 1 -- 172.21.15.64:0/2206196073 wait complete. 2023-11-24T06:30:14.180 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2023-11-24T06:30:13.146091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220848,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753872,"statfs":{"total":773094113280,"available":772867964928,"internally_reserved":0,"allocated":4792320,"data_stored":2599216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003654"},"pg_stats":[{"pgid":"1.0","version":"39'119","reported_seq":191,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-24T06:30:01.411310+0000","last_change":"2023-11-24T06:30:01.411376+0000","last_active":"2023-11-24T06:30:01.411310+0000","last_peered":"2023-11-24T06:30:01.411310+0000","last_clean":"2023-11-24T06:30:01.411310+0000","last_became_active":"2023-11-24T06:27:36.911727+0000","last_became_peered":"2023-11-24T06:27:36.911727+0000","last_unstale":"2023-11-24T06:30:01.411310+0000","last_undegraded":"2023-11-24T06:30:01.411310+0000","last_fullsized":"2023-11-24T06:30:01.411310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'119","last_scrub_stamp":"2023-11-24T06:30:01.411105+0000","last_deep_scrub":"39'119","last_deep_scrub_stamp":"2023-11-24T06:30:01.411105+0000","last_clean_scrub_stamp":"2023-11-24T06:30:01.411105+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:45:25.323481+0000","scrub_duration":0.011888345,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97399999999999998,"15min":0.97699999999999998},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.633,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.6639999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0469999999999999,"15min":1.0589999999999999},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7669999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.151}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0529999999999999,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.49},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.141,"15min":1.169},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.581,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0960000000000001,"15min":1.091},"min":{"1min":0.64100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.1779999999999999,"15min":1.1859999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.3720000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.145,"15min":1.149},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.319},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.347,"15min":1.369},"min":{"1min":0.65300000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.871,"5min":2.024,"15min":2.024},"last":1.579}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.758,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3380000000000001,"15min":1.343},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.044}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.379,"15min":1.383},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.093,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.56599999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.853,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.476,"15min":1.468},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.351},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.397,"15min":1.393},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4359999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.526,"5min":2.323,"15min":2.323},"last":1.091},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97299999999999998,"15min":0.96999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.1499999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.0700000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5669999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.526},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.048,"15min":1.0329999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.131,"15min":1.1259999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.619}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.151,"15min":1.167},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.228},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2290000000000001,"15min":1.2470000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.4179999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.288,"15min":1.302},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.379,"15min":1.3919999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.847,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.425,"15min":1.446},"min":{"1min":0.77900000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.887,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.417,"15min":1.4370000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8440000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.5289999999999999,"15min":1.5549999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.839,"5min":3.052,"15min":3.052},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.732,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.274}]}]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.53100000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.1120000000000001,"15min":1.1279999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.671,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.2290000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.105,"15min":1.1140000000000001},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.6439999999999999,"5min":1.79,"15min":1.79},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.224,"15min":1.24},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8320000000000001,"5min":1.857,"15min":1.857},"last":0.94499999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.222,"15min":1.24},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.897,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.677},{"interface":"front","average":{"1min":1.107,"5min":1.2589999999999999,"15min":1.2849999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6499999999999999,"5min":1.986,"15min":1.986},"last":1.472}]},{"osd":3,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.849,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.504},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.329,"15min":1.361},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.746,"5min":2.359,"15min":2.359},"last":1.3400000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.3260000000000001,"15min":1.3580000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.702,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.117},{"interface":"front","average":{"1min":1.327,"5min":1.377,"15min":1.385},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1160000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.4299999999999999,"15min":1.46},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9630000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.464,"15min":1.488},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9610000000000001,"5min":3.0299999999999998,"15min":3.0299999999999998},"last":1.6339999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5249999999999999,"15min":1.5389999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2570000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.722},{"interface":"front","average":{"1min":1.381,"5min":1.5880000000000001,"15min":1.6220000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1970000000000001,"5min":2.552,"15min":2.552},"last":1.863}]}]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.113,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.367},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2210000000000001,"15min":1.2370000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.446}]},{"osd":1,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64600000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.8879999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1459999999999999,"15min":1.1479999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.978,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6870000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.244},"min":{"1min":0.69999999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0840000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.218}]},{"osd":3,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.371},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2480000000000002,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.2130000000000001,"15min":1.23},"min":{"1min":0.58399999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0659999999999998,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.454,"15min":1.474},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2400000000000002,"5min":4.3979999999999997,"15min":4.3979999999999997},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.51,"15min":1.53},"min":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.157,"5min":4.2060000000000004,"15min":4.2060000000000004},"last":1.486}]},{"osd":6,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.381,"15min":1.401},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8220000000000001,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5580000000000001,"15min":1.577},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.3029999999999999,"5min":4.335,"15min":4.335},"last":1.524}]},{"osd":7,"last update":"Fri Nov 24 06:29:35 2023","interfaces":[{"interface":"back","average":{"1min":3.7959999999999998,"5min":2.0779999999999998,"15min":1.792},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":40.311999999999998,"5min":40.311999999999998,"15min":40.311999999999998},"last":1.6020000000000001},{"interface":"front","average":{"1min":3.7799999999999998,"5min":2.0840000000000001,"15min":1.802},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":40.286000000000001,"5min":40.286000000000001,"15min":40.286000000000001},"last":1.5580000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83499999999999996,"15min":0.83799999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.4019999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.288,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.91400000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.99299999999999999,"15min":1.014},"min":{"1min":0.501,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4530000000000001,"5min":1.629,"15min":1.629},"last":1.016},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.107,"15min":1.143},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.401,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.2290000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2210000000000001,"15min":1.2250000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.177}]},{"osd":5,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5960000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2829999999999999,"15min":1.288},"min":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.8500000000000001,"5min":1.885,"15min":1.885},"last":1.2909999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":0.68200000000000005,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7849999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.349,"15min":1.3700000000000001},"min":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9550000000000001,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.516}]},{"osd":7,"last update":"Fri Nov 24 06:29:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.421,"15min":1.4410000000000001},"min":{"1min":0.76800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.581},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9079999999999999,"15min":1.613},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":40.671999999999997,"5min":40.671999999999997,"15min":40.671999999999997},"last":1.3240000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91600000000000004,"15min":0.93600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.5469999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.2809999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.024},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3169999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.032},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.044,"15min":1.0780000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.569,"5min":2.278,"15min":2.278},"last":1.1279999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.353,"5min":1.847,"15min":1.847},"last":1.331},{"interface":"front","average":{"1min":1.032,"5min":1.232,"15min":1.2649999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7130000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.698}]},{"osd":4,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.323,"15min":1.345},"min":{"1min":0.69099999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7450000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.2090000000000001,"15min":1.2350000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.712,"5min":3.3839999999999999,"15min":3.3839999999999999},"last":1.6000000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.345,"15min":1.361},"min":{"1min":0.87,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.821,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8480000000000001,"5min":3.706,"15min":3.706},"last":1.409}]},{"osd":6,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2869999999999999,"15min":1.319},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7729999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4019999999999999,"15min":1.421},"min":{"1min":0.872,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.913,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.5469999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.46,"15min":1.5029999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.99,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.518},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.4119999999999999,"15min":1.4590000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.9910000000000001,"5min":3.661,"15min":3.661},"last":1.744}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":177310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96499999999999997,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3160000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.016,"15min":1.026},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.248}]},{"osd":2,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94499999999999995,"15min":0.96199999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.034,"15min":1.052},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.534,"5min":1.998,"15min":1.998},"last":1.52}]},{"osd":3,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.1140000000000001,"15min":1.137},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7589999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.028,"5min":1.079,"15min":1.087},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6870000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.5640000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.226,"15min":1.26},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4650000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3720000000000001,"15min":1.3759999999999999},"min":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.905,"5min":2.415,"15min":2.415},"last":1.397}]},{"osd":5,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.365,"15min":1.3979999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.8360000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.4590000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.47,"15min":1.48},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1259999999999999,"5min":2.544,"15min":2.544},"last":1.694},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.4630000000000001,"15min":1.4890000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.909,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.488,"15min":1.504},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0270000000000001,"5min":2.355,"15min":2.355},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5329999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.056,"5min":2.492,"15min":2.492},"last":1.76}]}]},{"osd":0,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1044480,"data_stored":767678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.94699999999999995,"15min":0.98699999999999999},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.133,"5min":2.54,"15min":2.54},"last":0.624},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.95199999999999996,"15min":0.98799999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.599,"5min":1.607,"15min":1.607},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.0840000000000001,"15min":1.1319999999999999},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.2609999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.91200000000000003,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.107,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.3899999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.123},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4410000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.173,"15min":1.206},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5700000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.7210000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3660000000000001,"15min":1.403},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5209999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.494,"15min":1.542},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5209999999999999,"5min":2.456,"15min":2.456},"last":1.2090000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3999999999999999,"15min":1.427},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.619},{"interface":"front","average":{"1min":1.028,"5min":1.4390000000000001,"15min":1.5069999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.673}]},{"osd":6,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.587,"15min":1.665},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7470000000000001,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.5429999999999999,"15min":1.591},"min":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.8879999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.468}]},{"osd":7,"last update":"Fri Nov 24 06:29:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.4510000000000001,"15min":1.5089999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7929999999999999,"5min":2.802,"15min":2.802},"last":1.51},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.631,"15min":1.698},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.8859999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:30:14.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T06:30:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.397+0000 7f60d8e9f640 1 Processor -- start 2023-11-24T06:30:14.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.397+0000 7f60d8e9f640 1 -- start start 2023-11-24T06:30:14.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.398+0000 7f60d8e9f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60d415d1d0 0x7f60d40d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.398+0000 7f60d8e9f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 0x7f60d40d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.398+0000 7f60d8e9f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d40d0090 con 0x7f60d415d5b0 2023-11-24T06:30:14.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.398+0000 7f60d8e9f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d40d02e0 con 0x7f60d415d1d0 2023-11-24T06:30:14.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.398+0000 7f60d8e9f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40d05c0 con 0x7f60d40b1f70 2023-11-24T06:30:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.398+0000 7f60d3fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f60d415d5b0 0x7f60d40cdb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55160/0 (socket says 172.21.15.64:55160) 2023-11-24T06:30:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.398+0000 7f60d3fff640 1 -- 172.21.15.64:0/965349372 learned_addr learned my addr 172.21.15.64:0/965349372 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T06:30:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.399+0000 7f60d37fe640 1 --2- 172.21.15.64:0/965349372 >> v2:172.21.15.162:3300/0 conn(0x7f60d415d1d0 0x7f60d40d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.399+0000 7f60d2ffd640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 0x7f60d40d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.399+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865048339 0 0) 0x7f60d40d0090 con 0x7f60d415d5b0 2023-11-24T06:30:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.399+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60c0003520 con 0x7f60d415d5b0 2023-11-24T06:30:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.399+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2722838747 0 0) 0x7f60c0003520 con 0x7f60d415d5b0 2023-11-24T06:30:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.399+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 msgr2=0x7f60d40d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.399+0000 7f60d27fc640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 0x7f60d40d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 >> v2:172.21.15.162:3300/0 conn(0x7f60d415d1d0 msgr2=0x7f60d40d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 --2- 172.21.15.64:0/965349372 >> v2:172.21.15.162:3300/0 conn(0x7f60d415d1d0 0x7f60d40d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40d08a0 con 0x7f60d415d5b0 2023-11-24T06:30:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2724280517 0 0) 0x7f60cc001670 con 0x7f60d415d5b0 2023-11-24T06:30:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 >> v1:172.21.15.64:6789/0 conn(0x7f60d415d5b0 legacy=0x7f60d40cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 0x7f60c0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0003ef0 0x7f60c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60c0004d00 0x7f60c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40d02e0 con 0x7f60c0004d00 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40d05c0 con 0x7f60c0003ef0 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c000b980 con 0x7f60d40b1f70 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d3fff640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60c0004d00 0x7f60c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d2ffd640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0003ef0 0x7f60c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.400+0000 7f60d37fe640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 0x7f60c0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.401+0000 7f60d3fff640 1 -- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 msgr2=0x7f60c0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.401+0000 7f60d3fff640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40b1f70 0x7f60c0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.401+0000 7f60d3fff640 1 -- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0003ef0 msgr2=0x7f60c00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.401+0000 7f60d3fff640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0003ef0 0x7f60c00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.401+0000 7f60d3fff640 1 -- 172.21.15.64:0/965349372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0003520 con 0x7f60c0004d00 2023-11-24T06:30:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.402+0000 7f60d3fff640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60c0004d00 0x7f60c0005180 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c290 tx=0x7f60c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ab57b7876b8e94 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.402+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc005ad0 con 0x7f60c0004d00 2023-11-24T06:30:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.402+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc005c30 con 0x7f60c0004d00 2023-11-24T06:30:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.402+0000 7f60d27fc640 1 -- 172.21.15.64:0/965349372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc02c8c0 con 0x7f60c0004d00 2023-11-24T06:30:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.403+0000 7f60d8e9f640 1 -- 172.21.15.64:0/965349372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60c0004d00 msgr2=0x7f60c0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.403+0000 7f60d8e9f640 1 --2- 172.21.15.64:0/965349372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60c0004d00 0x7f60c0005180 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c290 tx=0x7f60c00057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.403+0000 7f60d3fff640 1 -- 172.21.15.64:0/965349372 reap_dead start 2023-11-24T06:30:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.403+0000 7f60d8e9f640 1 -- 172.21.15.64:0/965349372 shutdown_connections 2023-11-24T06:30:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.403+0000 7f60d8e9f640 1 -- 172.21.15.64:0/965349372 >> 172.21.15.64:0/965349372 conn(0x7f60d40d3050 msgr2=0x7f60d40d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.403+0000 7f60d8e9f640 1 -- 172.21.15.64:0/965349372 shutdown_connections 2023-11-24T06:30:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.403+0000 7f60d8e9f640 1 -- 172.21.15.64:0/965349372 wait complete. 2023-11-24T06:30:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.405+0000 7f60d8e9f640 1 Processor -- start 2023-11-24T06:30:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.405+0000 7f60d8e9f640 1 -- start start 2023-11-24T06:30:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d8e9f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0004d00 0x7f60d40a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d8e9f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d41509a0 0x7f60d4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d8e9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d409def0 0x7f60d409e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d8e9f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d41516d0 con 0x7f60d409def0 2023-11-24T06:30:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d8e9f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d41519b0 con 0x7f60c0004d00 2023-11-24T06:30:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d8e9f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4151c90 con 0x7f60d41509a0 2023-11-24T06:30:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d409def0 0x7f60d409e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d409def0 0x7f60d409e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39432/0 (socket says 172.21.15.64:39432) 2023-11-24T06:30:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d3fff640 1 -- 172.21.15.64:0/3709136030 learned_addr learned my addr 172.21.15.64:0/3709136030 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T06:30:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d3fff640 1 -- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d41509a0 msgr2=0x7f60d4150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d3fff640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d41509a0 0x7f60d4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d3fff640 1 -- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0004d00 msgr2=0x7f60d40a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.406+0000 7f60d37fe640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0004d00 0x7f60d40a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60d3fff640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0004d00 0x7f60d40a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60d3fff640 1 -- 172.21.15.64:0/3709136030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d4151f70 con 0x7f60d409def0 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60d37fe640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0004d00 0x7f60d40a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60d3fff640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d409def0 0x7f60d409e370 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f60cc000c00 tx=0x7f60cc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc03c040 con 0x7f60d409def0 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc02ccf0 con 0x7f60d409def0 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0072b0 con 0x7f60d409def0 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d4152250 con 0x7f60d409def0 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.407+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d40a0a70 con 0x7f60d409def0 2023-11-24T06:30:14.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.409+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d40cfe20 con 0x7f60d409def0 2023-11-24T06:30:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.409+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f60cc032070 con 0x7f60d409def0 2023-11-24T06:30:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.410+0000 7f60cbfff640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f60b406ae40 0x7f60b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.410+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f60cc0aa080 con 0x7f60d409def0 2023-11-24T06:30:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.411+0000 7f60d37fe640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f60b406ae40 0x7f60b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.411+0000 7f60d37fe640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f60b406ae40 0x7f60b406d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007440 tx=0x7f60c400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.415+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f60cc078e20 con 0x7f60d409def0 2023-11-24T06:30:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.609+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f60d40d27e0 con 0x7f60d409def0 2023-11-24T06:30:14.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.616+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f60cc047020 con 0x7f60d409def0 2023-11-24T06:30:14.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.618+0000 7f60cbfff640 1 -- 172.21.15.64:0/3709136030 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f60cc078e20 con 0x7f60d409def0 2023-11-24T06:30:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.620+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f60b406ae40 msgr2=0x7f60b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.620+0000 7f60d8e9f640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f60b406ae40 0x7f60b406d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007440 tx=0x7f60c400b040 comp rx=0 tx=0).stop 2023-11-24T06:30:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.620+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d409def0 msgr2=0x7f60d409e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.620+0000 7f60d8e9f640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d409def0 0x7f60d409e370 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f60cc000c00 tx=0x7f60cc001a00 comp rx=0 tx=0).stop 2023-11-24T06:30:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 shutdown_connections 2023-11-24T06:30:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:6832/1000290349,v1:172.21.15.64:6833/1000290349] conn(0x7f60b406ae40 0x7f60b406d300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d409def0 0x7f60d409e370 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d41509a0 0x7f60d4150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 --2- 172.21.15.64:0/3709136030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60c0004d00 0x7f60d40a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 >> 172.21.15.64:0/3709136030 conn(0x7f60d40d3050 msgr2=0x7f60d40cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:14.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 shutdown_connections 2023-11-24T06:30:14.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T06:30:14.621+0000 7f60d8e9f640 1 -- 172.21.15.64:0/3709136030 wait complete. 2023-11-24T06:30:14.642 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mgr daemons: {'finisher-volumes.complete_latency.avgcount'} 2023-11-24T06:30:14.644 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T06:30:14.644 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:30:14.644 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:14.663 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:30:14.664 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:30:14.664 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:14.723 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:30:14.724 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:30:14.724 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:14.748 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:30:14.749 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:30:14.749 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:14.808 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:30:14.809 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T06:30:14.809 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T06:30:14.809 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:14.854 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T06:30:14.854 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T06:30:14.854 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:14.913 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T06:30:14.914 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T06:30:14.914 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:15.000 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T06:30:15.001 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T06:30:15.001 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:15.072 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T06:30:15.072 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T06:30:15.072 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:15.138 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T06:30:15.138 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T06:30:15.139 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:15.210 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T06:30:15.211 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T06:30:15.211 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:15.276 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T06:30:15.276 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T06:30:15.276 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:15.323 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T06:30:15.323 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T06:30:15.324 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T06:30:15.324 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:16.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-11-24T06:30:21.426 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T06:30:21.426 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T06:30:21.427 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:21.509 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T06:30:21.509 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T06:30:21.510 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T06:30:21.510 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:21.559 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T06:30:21.569 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T06:30:21.570 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:21.598 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T06:30:21.598 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T06:30:21.598 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:21.669 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T06:30:21.670 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T06:30:21.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T06:30:21.792 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:30:21.792 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T06:30:21.989 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-11-24T06:30:22.063 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:30:22.064 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T06:30:22.233 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:30:22.234 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T06:30:22.400 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:30:22.401 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T06:30:22.572 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:30:22.572 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T06:30:22.823 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:30:22.823 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T06:30:22.968 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:30:22.969 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T06:30:23.114 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:30:23.114 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T06:30:23.260 INFO:tasks.ceph:Archiving mon data... 2023-11-24T06:30:23.261 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/data/mon.a.tgz 2023-11-24T06:30:23.262 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-11-24T06:30:23.327 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.eiZJe3Funl 2023-11-24T06:30:23.327 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.eiZJe3Funl 2023-11-24T06:30:23.514 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.eiZJe3Funl 2023-11-24T06:30:23.668 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.eiZJe3Funl is 295KB 2023-11-24T06:30:23.780 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.eiZJe3Funl 2023-11-24T06:30:23.843 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/data/mon.b.tgz 2023-11-24T06:30:23.844 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2023-11-24T06:30:23.897 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.jiqXjXmvSq 2023-11-24T06:30:23.897 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jiqXjXmvSq 2023-11-24T06:30:24.082 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.jiqXjXmvSq 2023-11-24T06:30:24.205 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.jiqXjXmvSq is 395KB 2023-11-24T06:30:24.312 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.jiqXjXmvSq 2023-11-24T06:30:24.366 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/data/mon.c.tgz 2023-11-24T06:30:24.366 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2023-11-24T06:30:24.411 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.7eIng3pVIv 2023-11-24T06:30:24.412 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.7eIng3pVIv 2023-11-24T06:30:24.622 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.7eIng3pVIv 2023-11-24T06:30:24.754 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.7eIng3pVIv is 395KB 2023-11-24T06:30:24.848 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.7eIng3pVIv 2023-11-24T06:30:24.905 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T06:30:24.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:30:24.916 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:30:25.019 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T06:30:25.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:30:25.028 DEBUG:teuthology.orchestra.run.smithi162:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:30:25.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:30:25.136 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:30:25.148 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T06:30:25.151 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/remote/smithi064/crash 2023-11-24T06:30:25.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:30:25.236 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/remote/smithi162/crash 2023-11-24T06:30:25.237 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:30:25.335 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T06:30:25.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:25.346 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:25.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:25.597 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:25.706 INFO:tasks.ceph:Compressing logs... 2023-11-24T06:30:25.707 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:30:25.716 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:30:25.809 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T06:30:25.810 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94328.log 2023-11-24T06:30:25.810 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98095.log 2023-11-24T06:30:25.811 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88509.log 2023-11-24T06:30:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96156.log 2023-11-24T06:30:25.812 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.1.94328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88309.log 2023-11-24T06:30:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99911.log 2023-11-24T06:30:25.813 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98095.log.gz 2023-11-24T06:30:25.814 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88509.log: 90.8% -- replaced with /var/log/ceph/ceph-client.1.94328.log.gz 2023-11-24T06:30:25.814 INFO:teuthology.orchestra.run.smithi162.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88509.log.gz 2023-11-24T06:30:25.814 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88389.log 2023-11-24T06:30:25.815 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88309.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88309.log.gz -5 2023-11-24T06:30:25.815 INFO:teuthology.orchestra.run.smithi162.stderr: --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:30:25.816 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.96156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102610.log 2023-11-24T06:30:25.817 INFO:teuthology.orchestra.run.smithi064.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.96156.log.gz 2023-11-24T06:30:25.817 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99911.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T06:30:25.817 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99911.log.gz 2023-11-24T06:30:25.817 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99617.log 2023-11-24T06:30:25.817 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102610.log.gz 2023-11-24T06:30:25.819 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99407.log 2023-11-24T06:30:25.819 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101019.log 2023-11-24T06:30:25.819 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99617.log.gz 2023-11-24T06:30:25.820 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88389.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T06:30:25.820 INFO:teuthology.orchestra.run.smithi162.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88389.log.gz 2023-11-24T06:30:25.820 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T06:30:25.820 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94845.log 2023-11-24T06:30:25.821 INFO:teuthology.orchestra.run.smithi162.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:30:25.821 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88269.log 2023-11-24T06:30:25.822 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103814.log 2023-11-24T06:30:25.822 INFO:teuthology.orchestra.run.smithi064.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99407.log.gz 2023-11-24T06:30:25.822 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101019.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101019.log.gz 2023-11-24T06:30:25.822 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95412.log 2023-11-24T06:30:25.825 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93586.log 2023-11-24T06:30:25.825 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103814.log.gz 2023-11-24T06:30:25.825 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95412.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95412.log.gz -5 2023-11-24T06:30:25.825 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94768.log 2023-11-24T06:30:25.826 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.3.94845.log: 91.8% -- replaced with /var/log/ceph/ceph-client.3.94845.log.gz 2023-11-24T06:30:25.826 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T06:30:25.827 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88269.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T06:30:25.827 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.7.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88269.log.gz 2023-11-24T06:30:25.827 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88549.log 2023-11-24T06:30:25.828 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88349.log 2023-11-24T06:30:25.830 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95100.log 2023-11-24T06:30:25.830 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93586.log.gz 2023-11-24T06:30:25.830 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94768.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94768.log.gz 2023-11-24T06:30:25.830 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102346.log 2023-11-24T06:30:25.831 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95100.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95100.log.gz 2023-11-24T06:30:25.831 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98363.log 2023-11-24T06:30:25.831 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102346.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102346.log.gz 2023-11-24T06:30:25.832 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98777.log 2023-11-24T06:30:25.833 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88549.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T06:30:25.833 INFO:teuthology.orchestra.run.smithi162.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88549.log.gz 2023-11-24T06:30:25.834 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88469.log 2023-11-24T06:30:25.834 INFO:teuthology.orchestra.run.smithi162.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88349.log.gz 2023-11-24T06:30:25.834 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88429.log 2023-11-24T06:30:25.835 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103105.log 2023-11-24T06:30:25.835 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98363.log.gz 2023-11-24T06:30:25.836 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98777.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98777.log.gz 2023-11-24T06:30:25.836 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103021.log 2023-11-24T06:30:25.836 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96822.log 2023-11-24T06:30:25.837 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103105.log.gz 2023-11-24T06:30:25.837 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103021.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103021.log.gz 2023-11-24T06:30:25.837 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98735.log 2023-11-24T06:30:25.840 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.96822.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96822.log.gz 2023-11-24T06:30:25.840 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98609.log 2023-11-24T06:30:25.840 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98735.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98735.log.gz 2023-11-24T06:30:25.840 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101365.log 2023-11-24T06:30:25.841 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98609.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98609.log.gz 2023-11-24T06:30:25.841 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97268.log 2023-11-24T06:30:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88469.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88469.log.gz 2023-11-24T06:30:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95590.log 2023-11-24T06:30:25.842 INFO:teuthology.orchestra.run.smithi162.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88429.log.gz 2023-11-24T06:30:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T06:30:25.842 INFO:teuthology.orchestra.run.smithi162.stderr: --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:30:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.1.95590.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-24T06:30:25.843 INFO:teuthology.orchestra.run.smithi162.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.1.95590.log.gz 2023-11-24T06:30:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:30:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T06:30:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T06:30:25.845 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95184.log 2023-11-24T06:30:25.845 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101365.log.gz 2023-11-24T06:30:25.845 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97268.log: gzip 88.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.97268.log.gz -- 2023-11-24T06:30:25.845 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.101155.log 2023-11-24T06:30:25.845 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94104.log 2023-11-24T06:30:25.845 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95184.log.gz 2023-11-24T06:30:25.845 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101155.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101155.log.gz 2023-11-24T06:30:25.846 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101834.log 2023-11-24T06:30:25.846 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T06:30:25.847 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104192.log 2023-11-24T06:30:25.847 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94104.log.gz 2023-11-24T06:30:25.848 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.101834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99287.log 2023-11-24T06:30:25.848 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104192.log: 89.3% -- replaced with /var/log/ceph/ceph-client.0.101834.log.gz 2023-11-24T06:30:25.849 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104192.log.gz 2023-11-24T06:30:25.849 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97832.log 2023-11-24T06:30:25.850 INFO:teuthology.orchestra.run.smithi162.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T06:30:25.851 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97520.log 2023-11-24T06:30:25.851 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99287.log.gz 2023-11-24T06:30:25.852 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97832.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97832.log.gz 2023-11-24T06:30:25.852 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100637.log 2023-11-24T06:30:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94099.log 2023-11-24T06:30:25.853 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97520.log.gz 2023-11-24T06:30:25.854 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94974.log 2023-11-24T06:30:25.855 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100637.log.gz 2023-11-24T06:30:25.855 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98693.log 2023-11-24T06:30:25.856 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94099.log.gz 2023-11-24T06:30:25.857 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100037.log 2023-11-24T06:30:25.857 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94974.log.gz 2023-11-24T06:30:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98693.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.98693.log.gz 2023-11-24T06:30:25.858 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93838.log 2023-11-24T06:30:25.859 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100889.log 2023-11-24T06:30:25.859 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100037.log.gz 2023-11-24T06:30:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99743.log 2023-11-24T06:30:25.861 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93838.log.gz 2023-11-24T06:30:25.862 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102466.log 2023-11-24T06:30:25.862 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100889.log.gz 2023-11-24T06:30:25.863 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98053.log 2023-11-24T06:30:25.863 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99743.log.gz 2023-11-24T06:30:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101113.log 2023-11-24T06:30:25.864 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102466.log.gz 2023-11-24T06:30:25.865 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98053.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82760.log 2023-11-24T06:30:25.865 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98053.log.gz 2023-11-24T06:30:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97058.log 2023-11-24T06:30:25.867 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101113.log.gz 2023-11-24T06:30:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.tmp-client.admin.82760.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82760.log.gz 2023-11-24T06:30:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98137.log 2023-11-24T06:30:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93108.log 2023-11-24T06:30:25.869 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97058.log.gz 2023-11-24T06:30:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94440.log 2023-11-24T06:30:25.870 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98137.log.gz 2023-11-24T06:30:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103209.log 2023-11-24T06:30:25.872 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.93108.log.gz 2023-11-24T06:30:25.872 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94096.log 2023-11-24T06:30:25.873 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94440.log.gz 2023-11-24T06:30:25.874 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98441.log 2023-11-24T06:30:25.874 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103209.log.gz 2023-11-24T06:30:25.875 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102895.log 2023-11-24T06:30:25.875 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94096.log.gz 2023-11-24T06:30:25.876 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97790.log 2023-11-24T06:30:25.876 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98441.log.gz 2023-11-24T06:30:25.877 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102811.log 2023-11-24T06:30:25.878 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102895.log.gz 2023-11-24T06:30:25.879 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103063.log 2023-11-24T06:30:25.879 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97790.log.gz 2023-11-24T06:30:25.880 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102811.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.102811.log.gz --verbose 2023-11-24T06:30:25.880 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.98279.log 2023-11-24T06:30:25.881 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97352.log 2023-11-24T06:30:25.881 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103063.log.gz 2023-11-24T06:30:25.882 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99119.log 2023-11-24T06:30:25.882 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98279.log.gz 2023-11-24T06:30:25.883 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98893.log 2023-11-24T06:30:25.884 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97352.log.gz 2023-11-24T06:30:25.885 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99119.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.99119.log.gz 2023-11-24T06:30:25.885 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100307.log 2023-11-24T06:30:25.886 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94426.log 2023-11-24T06:30:25.886 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98893.log.gz 2023-11-24T06:30:25.887 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93712.log 2023-11-24T06:30:25.887 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100307.log.gz 2023-11-24T06:30:25.888 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102099.log 2023-11-24T06:30:25.888 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94426.log.gz 2023-11-24T06:30:25.889 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102424.log 2023-11-24T06:30:25.890 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93712.log.gz 2023-11-24T06:30:25.891 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98525.log 2023-11-24T06:30:25.891 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102099.log.gz 2023-11-24T06:30:25.892 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97436.log 2023-11-24T06:30:25.893 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102424.log.gz 2023-11-24T06:30:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96864.log 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98525.log.gz 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93416.log 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97436.log.gz 2023-11-24T06:30:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.96864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98977.log 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96864.log.gz 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93416.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93416.log.gz 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94758.log 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100511.log 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98977.log.gz 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94758.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94758.log.gz 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94103.log 2023-11-24T06:30:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100511.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:30:25.902 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100511.log.gz 2023-11-24T06:30:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95496.log 2023-11-24T06:30:25.903 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94103.log.gz 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99449.log 2023-11-24T06:30:25.905 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100553.log 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95496.log.gz 2023-11-24T06:30:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99449.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99449.log.gz 2023-11-24T06:30:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98935.log 2023-11-24T06:30:25.909 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99491.log 2023-11-24T06:30:25.909 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100553.log.gz 2023-11-24T06:30:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100805.log 2023-11-24T06:30:25.910 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98935.log.gz 2023-11-24T06:30:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104276.log 2023-11-24T06:30:25.911 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99491.log.gz 2023-11-24T06:30:25.912 INFO:teuthology.orchestra.run.smithi162.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T06:30:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95454.log 2023-11-24T06:30:25.913 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100805.log.gz 2023-11-24T06:30:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104360.log 2023-11-24T06:30:25.914 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104276.log.gz 2023-11-24T06:30:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102979.log 2023-11-24T06:30:25.915 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95454.log.gz 2023-11-24T06:30:25.916 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95058.log 2023-11-24T06:30:25.916 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104360.log.gz 2023-11-24T06:30:25.917 INFO:teuthology.orchestra.run.smithi162.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T06:30:25.920 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101069.log 2023-11-24T06:30:25.920 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102979.log.gz 2023-11-24T06:30:25.920 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94436.log 2023-11-24T06:30:25.920 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95058.log.gz 2023-11-24T06:30:25.920 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101239.log 2023-11-24T06:30:25.921 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101069.log.gz 2023-11-24T06:30:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99161.log 2023-11-24T06:30:25.923 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94436.log.gz 2023-11-24T06:30:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102727.log 2023-11-24T06:30:25.923 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101239.log.gz 2023-11-24T06:30:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99161.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99161.log.gz 2023-11-24T06:30:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97748.log 2023-11-24T06:30:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100847.log 2023-11-24T06:30:25.925 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102727.log.gz 2023-11-24T06:30:25.926 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97748.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97748.log.gz 2023-11-24T06:30:25.926 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.104402.log 2023-11-24T06:30:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97394.log 2023-11-24T06:30:25.928 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100847.log.gz 2023-11-24T06:30:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104402.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104402.log.gz 2023-11-24T06:30:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94087.log 2023-11-24T06:30:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102262.log 2023-11-24T06:30:25.930 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97394.log.gz 2023-11-24T06:30:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97969.log 2023-11-24T06:30:25.931 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94087.log.gz 2023-11-24T06:30:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95370.log 2023-11-24T06:30:25.932 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102262.log.gz 2023-11-24T06:30:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94102.log 2023-11-24T06:30:25.934 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97969.log.gz 2023-11-24T06:30:25.935 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95370.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95370.log.gz 2023-11-24T06:30:25.935 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103772.log 2023-11-24T06:30:25.936 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92931.log 2023-11-24T06:30:25.937 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94102.log.gz 2023-11-24T06:30:25.938 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103772.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103772.log.gz 2023-11-24T06:30:25.938 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104108.log 2023-11-24T06:30:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.92931.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92931.log.gz 2023-11-24T06:30:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102178.log 2023-11-24T06:30:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98321.log 2023-11-24T06:30:25.940 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104108.log.gz 2023-11-24T06:30:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104444.log 2023-11-24T06:30:25.942 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102178.log.gz 2023-11-24T06:30:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100349.log 2023-11-24T06:30:25.943 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98321.log.gz 2023-11-24T06:30:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104444.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104444.log.gz 2023-11-24T06:30:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97562.log 2023-11-24T06:30:25.946 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95622.log 2023-11-24T06:30:25.946 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100349.log.gz 2023-11-24T06:30:25.947 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100223.log 2023-11-24T06:30:25.947 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97562.log.gz 2023-11-24T06:30:25.948 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99203.log 2023-11-24T06:30:25.949 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.95622.log.gz 2023-11-24T06:30:25.950 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100223.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T06:30:25.950 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100223.log.gz 2023-11-24T06:30:25.951 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99203.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99203.log.gz 2023-11-24T06:30:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95226.log 2023-11-24T06:30:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93880.log 2023-11-24T06:30:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99785.log 2023-11-24T06:30:25.954 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95226.log.gz 2023-11-24T06:30:25.955 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100679.log 2023-11-24T06:30:25.956 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93880.log.gz 2023-11-24T06:30:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99785.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99785.log.gz 2023-11-24T06:30:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97874.log 2023-11-24T06:30:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99061.log 2023-11-24T06:30:25.959 INFO:teuthology.orchestra.run.smithi162.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T06:30:25.960 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100679.log.gz 2023-11-24T06:30:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97874.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97874.log.gz 2023-11-24T06:30:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97604.log 2023-11-24T06:30:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94425.log 2023-11-24T06:30:25.963 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99061.log.gz 2023-11-24T06:30:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97604.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97604.log.gz 2023-11-24T06:30:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103430.log 2023-11-24T06:30:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102568.log 2023-11-24T06:30:25.966 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.94425.log.gz 2023-11-24T06:30:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103430.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103430.log.gz 2023-11-24T06:30:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96718.log 2023-11-24T06:30:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93322.log 2023-11-24T06:30:25.969 INFO:teuthology.orchestra.run.smithi162.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T06:30:25.969 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102568.log.gz 2023-11-24T06:30:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.2.96718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97184.log 2023-11-24T06:30:25.971 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.2.96718.log.gz 2023-11-24T06:30:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94766.log 2023-11-24T06:30:25.972 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93322.log.gz 2023-11-24T06:30:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97184.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97184.log.gz 2023-11-24T06:30:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104318.log 2023-11-24T06:30:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99365.log 2023-11-24T06:30:25.975 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94766.log.gz 2023-11-24T06:30:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104318.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104318.log.gz 2023-11-24T06:30:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99701.log 2023-11-24T06:30:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102508.log 2023-11-24T06:30:25.978 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99365.log.gz 2023-11-24T06:30:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99701.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99701.log.gz 2023-11-24T06:30:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98819.log 2023-11-24T06:30:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99245.log 2023-11-24T06:30:25.982 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102508.log.gz 2023-11-24T06:30:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98819.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98819.log.gz 2023-11-24T06:30:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102220.log 2023-11-24T06:30:25.985 INFO:teuthology.orchestra.run.smithi162.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T06:30:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97100.log 2023-11-24T06:30:25.986 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99245.log.gz 2023-11-24T06:30:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102220.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102220.log.gz 2023-11-24T06:30:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104024.log 2023-11-24T06:30:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101557.log 2023-11-24T06:30:25.988 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97100.log.gz 2023-11-24T06:30:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95016.log 2023-11-24T06:30:25.990 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104024.log.gz 2023-11-24T06:30:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101557.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101557.log.gz 2023-11-24T06:30:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98195.log 2023-11-24T06:30:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104547.log 2023-11-24T06:30:25.993 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95016.log.gz 2023-11-24T06:30:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98195.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98195.log.gz 2023-11-24T06:30:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97016.log 2023-11-24T06:30:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100139.log 2023-11-24T06:30:25.996 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104547.log.gz 2023-11-24T06:30:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97016.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97016.log.gz 2023-11-24T06:30:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T06:30:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94422.log 2023-11-24T06:30:26.002 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100139.log.gz 2023-11-24T06:30:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101281.log 2023-11-24T06:30:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94422.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94422.log.gz 2023-11-24T06:30:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103688.log 2023-11-24T06:30:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94437.log 2023-11-24T06:30:26.004 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101281.log.gz 2023-11-24T06:30:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103688.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103688.log.gz 2023-11-24T06:30:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102937.log 2023-11-24T06:30:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98483.log 2023-11-24T06:30:26.007 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94437.log.gz 2023-11-24T06:30:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102937.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102937.log.gz 2023-11-24T06:30:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103165.log 2023-11-24T06:30:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103898.log 2023-11-24T06:30:26.010 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98483.log.gz 2023-11-24T06:30:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102685.log 2023-11-24T06:30:26.011 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103165.log.gz 2023-11-24T06:30:26.013 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99533.log 2023-11-24T06:30:26.013 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103898.log.gz 2023-11-24T06:30:26.014 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102685.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102685.log.gz 2023-11-24T06:30:26.015 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102853.log 2023-11-24T06:30:26.016 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99533.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.93544.log -- replaced with /var/log/ceph/ceph-client.admin.99533.log.gz 2023-11-24T06:30:26.016 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T06:30:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101973.log 2023-11-24T06:30:26.018 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102853.log.gz 2023-11-24T06:30:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93544.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93544.log.gz 2023-11-24T06:30:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95724.log 2023-11-24T06:30:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99995.log 2023-11-24T06:30:26.021 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101973.log.gz 2023-11-24T06:30:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95724.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95724.log.gz 2023-11-24T06:30:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93796.log 2023-11-24T06:30:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99995.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99995.log.gz 2023-11-24T06:30:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94095.log 2023-11-24T06:30:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100429.log 2023-11-24T06:30:26.026 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93796.log.gz 2023-11-24T06:30:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94095.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94095.log.gz 2023-11-24T06:30:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94717.log 2023-11-24T06:30:26.029 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100429.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100429.log.gz 2023-11-24T06:30:26.030 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94439.log 2023-11-24T06:30:26.031 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93922.log 2023-11-24T06:30:26.031 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94717.log.gz 2023-11-24T06:30:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94439.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94439.log.gz 2023-11-24T06:30:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95328.log 2023-11-24T06:30:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93027.log 2023-11-24T06:30:26.034 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93922.log.gz 2023-11-24T06:30:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95328.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95328.log.gz 2023-11-24T06:30:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102769.log 2023-11-24T06:30:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93027.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93027.log.gz 2023-11-24T06:30:26.038 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93277.log 2023-11-24T06:30:26.039 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102769.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T06:30:26.039 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102769.log.gz 2023-11-24T06:30:26.042 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93277.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93277.log.gz 2023-11-24T06:30:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99827.log 2023-11-24T06:30:26.043 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T06:30:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103940.log 2023-11-24T06:30:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103259.log 2023-11-24T06:30:26.045 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99827.log.gz 2023-11-24T06:30:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103940.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103940.log.gz 2023-11-24T06:30:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95538.log 2023-11-24T06:30:26.046 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98567.log 2023-11-24T06:30:26.047 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103259.log.gz 2023-11-24T06:30:26.047 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95538.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95538.log.gz 2023-11-24T06:30:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97478.log 2023-11-24T06:30:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103982.log 2023-11-24T06:30:26.050 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98567.log.gz 2023-11-24T06:30:26.051 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97478.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97478.log.gz 2023-11-24T06:30:26.051 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102057.log 2023-11-24T06:30:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102015.log 2023-11-24T06:30:26.053 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103982.log.gz 2023-11-24T06:30:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102057.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102057.log.gz 2023-11-24T06:30:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101491.log 2023-11-24T06:30:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100265.log 2023-11-24T06:30:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102015.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102015.log.gz 2023-11-24T06:30:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103730.log 2023-11-24T06:30:26.057 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.101491.log.gz 2023-11-24T06:30:26.058 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98237.log 2023-11-24T06:30:26.058 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100265.log.gz 2023-11-24T06:30:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103730.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103730.log.gz 2023-11-24T06:30:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104587.log 2023-11-24T06:30:26.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93460.log/var/log/ceph/ceph-client.admin.98237.log: 2023-11-24T06:30:26.069 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98237.log.gz 2023-11-24T06:30:26.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104587.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104587.log.gz 2023-11-24T06:30:26.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89103.log 2023-11-24T06:30:26.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103304.log 2023-11-24T06:30:26.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.89103.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93460.log.gz 2023-11-24T06:30:26.070 INFO:teuthology.orchestra.run.smithi064.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.89103.log.gz 2023-11-24T06:30:26.070 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95142.log 2023-11-24T06:30:26.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103304.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103304.log.gz 2023-11-24T06:30:26.070 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101197.log 2023-11-24T06:30:26.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100721.log 2023-11-24T06:30:26.071 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95142.log.gz 2023-11-24T06:30:26.071 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101197.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101197.log.gz 2023-11-24T06:30:26.071 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99019.log 2023-11-24T06:30:26.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100721.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100721.log.gz 2023-11-24T06:30:26.072 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103388.log 2023-11-24T06:30:26.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99019.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99019.log.gz 2023-11-24T06:30:26.072 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99869.log 2023-11-24T06:30:26.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100181.log 2023-11-24T06:30:26.073 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103388.log.gz 2023-11-24T06:30:26.079 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99869.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99869.log.gz 2023-11-24T06:30:26.079 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99659.log 2023-11-24T06:30:26.079 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100181.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100181.log.gzgzip 2023-11-24T06:30:26.079 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T06:30:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99659.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99659.log.gz 2023-11-24T06:30:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95580.log 2023-11-24T06:30:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97310.log 2023-11-24T06:30:26.081 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T06:30:26.081 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95580.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95580.log.gz 2023-11-24T06:30:26.081 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100079.log 2023-11-24T06:30:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97310.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97310.log.gz -5 2023-11-24T06:30:26.085 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.104486.log 2023-11-24T06:30:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100079.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100079.log.gz 2023-11-24T06:30:26.085 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102304.log 2023-11-24T06:30:26.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104486.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104486.log.gz 2023-11-24T06:30:26.086 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94761.log 2023-11-24T06:30:26.088 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102304.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102304.log.gz 2023-11-24T06:30:26.088 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93628.log 2023-11-24T06:30:26.088 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94761.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94761.log.gz 2023-11-24T06:30:26.089 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101323.log 2023-11-24T06:30:26.090 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93628.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gzgzip 2023-11-24T06:30:26.090 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100763.log 2023-11-24T06:30:26.091 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93628.log.gz 2023-11-24T06:30:26.091 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101323.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101323.log.gz 2023-11-24T06:30:26.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103856.log 2023-11-24T06:30:26.092 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94092.log 2023-11-24T06:30:26.093 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100763.log.gz 2023-11-24T06:30:26.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103856.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103856.log.gz 2023-11-24T06:30:26.094 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:30:26.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94431.log 2023-11-24T06:30:26.095 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94092.log.gz 2023-11-24T06:30:26.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98011.log 2023-11-24T06:30:26.097 INFO:teuthology.orchestra.run.smithi064.stderr: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:30:26.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94431.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T06:30:26.098 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94431.log.gz 2023-11-24T06:30:26.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98011.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98011.log.gz 2023-11-24T06:30:26.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100931.log 2023-11-24T06:30:26.100 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T06:30:26.101 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100931.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100931.log.gz 2023-11-24T06:30:26.101 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101407.log 2023-11-24T06:30:26.103 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101449.log 2023-11-24T06:30:26.103 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T06:30:26.104 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104150.log 2023-11-24T06:30:26.104 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101407.log.gz 2023-11-24T06:30:26.105 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93754.log 2023-11-24T06:30:26.106 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101449.log.gz 2023-11-24T06:30:26.107 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104150.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104150.log.gz 2023-11-24T06:30:26.107 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100595.log 2023-11-24T06:30:26.108 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93754.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.96974.log -- replaced with /var/log/ceph/ceph-client.admin.93754.log.gz 2023-11-24T06:30:26.109 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T06:30:26.110 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100595.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100595.log.gz 2023-11-24T06:30:26.110 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94777.log 2023-11-24T06:30:26.111 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.96974.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96974.log.gz 2023-11-24T06:30:26.112 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93168.log 2023-11-24T06:30:26.112 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99575.log 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94777.log.gz 2023-11-24T06:30:26.114 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93168.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93168.log.gz 2023-11-24T06:30:26.114 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88831.log 2023-11-24T06:30:26.115 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99575.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99575.log.gz 2023-11-24T06:30:26.116 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99953.log 2023-11-24T06:30:26.117 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.88831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93221.log 2023-11-24T06:30:26.118 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99953.log: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.88831.log.gz 2023-11-24T06:30:26.118 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99953.log.gz 2023-11-24T06:30:26.118 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98651.log 2023-11-24T06:30:26.120 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93221.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93221.log.gz 2023-11-24T06:30:26.120 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103346.log 2023-11-24T06:30:26.121 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96312.log 2023-11-24T06:30:26.122 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98651.log.gz 2023-11-24T06:30:26.123 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103346.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103346.log.gz 2023-11-24T06:30:26.123 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97688.log 2023-11-24T06:30:26.124 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.96312.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.96312.log.gz 2023-11-24T06:30:26.124 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103646.log 2023-11-24T06:30:26.125 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97688.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97688.log.gz 2023-11-24T06:30:26.126 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97646.log 2023-11-24T06:30:26.127 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104066.log 2023-11-24T06:30:26.127 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103646.log.gz 2023-11-24T06:30:26.128 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97646.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97646.log.gz 2023-11-24T06:30:26.129 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94748.log 2023-11-24T06:30:26.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104066.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104066.log.gz 2023-11-24T06:30:26.130 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93670.log 2023-11-24T06:30:26.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97226.log 2023-11-24T06:30:26.132 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94748.log.gz 2023-11-24T06:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93670.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93670.log.gz 2023-11-24T06:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103472.log 2023-11-24T06:30:26.134 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97226.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97226.log.gz 2023-11-24T06:30:26.134 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94778.log 2023-11-24T06:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95286.log 2023-11-24T06:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103472.log.gz 2023-11-24T06:30:26.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94778.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94778.log.gz 2023-11-24T06:30:26.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89233.log 2023-11-24T06:30:26.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95286.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95286.log.gz 2023-11-24T06:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97142.log 2023-11-24T06:30:26.140 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.89233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104234.log 2023-11-24T06:30:26.140 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T06:30:26.140 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89233.log.gz 2023-11-24T06:30:26.141 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97142.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97142.log.gz 2023-11-24T06:30:26.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100975.log 2023-11-24T06:30:26.143 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104234.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104234.log.gz 2023-11-24T06:30:26.143 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93364.log 2023-11-24T06:30:26.144 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100975.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100975.log.gz 2023-11-24T06:30:26.144 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T06:30:26.145 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93364.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93364.log.gz 2023-11-24T06:30:26.146 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93502.log 2023-11-24T06:30:26.147 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.93502.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93502.log.gz 2023-11-24T06:30:26.157 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T06:30:26.197 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T06:30:36.255 INFO:teuthology.orchestra.run.smithi162.stderr: 96.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T06:30:36.259 INFO:teuthology.orchestra.run.smithi162.stderr: 2023-11-24T06:30:36.259 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m10.490s 2023-11-24T06:30:36.259 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m10.151s 2023-11-24T06:30:36.259 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m1.074s 2023-11-24T06:30:40.776 INFO:teuthology.orchestra.run.smithi064.stderr: 96.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T06:30:40.781 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T06:30:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m15.015s 2023-11-24T06:30:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m14.187s 2023-11-24T06:30:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m2.015s 2023-11-24T06:30:40.783 INFO:tasks.ceph:Archiving logs... 2023-11-24T06:30:40.783 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/remote/smithi064/log 2023-11-24T06:30:40.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:30:42.459 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/remote/smithi162/log 2023-11-24T06:30:42.460 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:30:43.627 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T06:30:43.638 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mgr daemons: {'finisher-volumes.complete_latency.avgcount'} 2023-11-24T06:30:43.639 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T06:30:43.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:30:43.649 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:30:43.813 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T06:30:43.822 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T06:30:43.823 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:30:43.832 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:30:43.884 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:30:43.890 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:30:43.894 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:30:43.894 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-11-24T06:30:43.894 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +1772us[+1768us] +/- 53ms 2023-11-24T06:30:43.895 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -2051us[-2051us] +/- 73ms 2023-11-24T06:30:43.895 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -451us[ -451us] +/- 72ms 2023-11-24T06:30:43.895 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:30:43.900 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:30:43.900 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2023-11-24T06:30:43.900 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +1788us[+1793us] +/- 53ms 2023-11-24T06:30:43.901 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -2065us[-2060us] +/- 73ms 2023-11-24T06:30:43.901 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -452us[ -447us] +/- 72ms 2023-11-24T06:30:43.901 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:30:43.902 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T06:30:43.911 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T06:30:43.912 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T06:30:43.922 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:30:44.203 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:30:44.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:30:44.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:30:44.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:30:44.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:30:44.691 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T06:30:44.702 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T06:30:44.742 INFO:teuthology.task.internal:Duration was 993.465331 seconds 2023-11-24T06:30:44.743 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T06:30:44.752 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T06:30:44.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:30:44.762 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:30:44.932 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:30:44.942 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:30:45.271 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T06:30:45.271 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T06:30:45.272 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:30:45.342 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2023-11-24T06:30:45.343 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:30:45.423 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T06:30:45.423 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:30:45.433 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:30:45.531 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T06:30:45.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:30:45.538 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:30:45.724 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T06:30:45.742 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T06:30:45.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:30:45.844 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:30:45.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T06:30:45.959 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:30:45.969 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:30:46.080 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2023-11-24T06:30:46.083 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2023-11-24T06:30:46.137 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:30:46.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:30:46.192 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:30:46.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:30:46.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T06:30:46.264 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T06:30:46.265 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/remote/smithi064 2023-11-24T06:30:46.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:30:46.390 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558/remote/smithi162 2023-11-24T06:30:46.390 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:30:46.505 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T06:30:46.505 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:30:46.522 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:30:46.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T06:30:46.600 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T06:30:46.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T06:30:46.611 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T06:30:46.611 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:30:46.621 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:30:46.673 INFO:teuthology.orchestra.run.smithi064.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:30 /home/ubuntu/cephtest 2023-11-24T06:30:46.681 INFO:teuthology.orchestra.run.smithi162.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:30 /home/ubuntu/cephtest 2023-11-24T06:30:46.684 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T06:30:46.695 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T06:30:46.707 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: The following counters failed to be set on mgr daemons: {'finisher-volumes.complete_latency.avgcount'} 2023-11-24T06:30:46.708 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T06:30:46.735 DEBUG:teuthology.nuke:shortname: smithi064 2023-11-24T06:30:46.735 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:30:46.758 DEBUG:teuthology.nuke:shortname: smithi162 2023-11-24T06:30:46.758 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:30:46.762 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:04:13.066076', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMBWw4LPwMQvxVPgW6Z5nJVeimE42KGyRzyRz7kIiXdPqs+1FRULDPryrnIqWH/SVfFlOo5bSnajqLLE+P57dUU='} 2023-11-24T06:30:46.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465558', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:04:13.067676', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWKYd2+qmve4xjdoRtS92UFJvCTKl8gtKVkGgl5I4mgJOj2juuxiBXVDTfPPdQh3E8RHK5Ef3PVr73pU1bGv8='} 2023-11-24T06:30:46.814 INFO:teuthology.orchestra.console.smithi064:Power off 2023-11-24T06:30:46.815 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:30:46.842 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2023-11-24T06:30:46.842 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:30:46.861 INFO:teuthology.orchestra.console.smithi162:Power off 2023-11-24T06:30:46.861 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:30:46.877 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:30:46.917 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-11-24T06:30:50.879 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:30:50.918 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:30:50.991 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-11-24T06:30:51.030 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-11-24T06:30:54.995 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:30:55.031 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:30:55.106 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2023-11-24T06:30:55.142 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-11-24T06:30:55.207 INFO:teuthology.orchestra.console.smithi064:Power off completed 2023-11-24T06:30:55.243 INFO:teuthology.orchestra.console.smithi162:Power off completed 2023-11-24T06:30:55.349 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} duration: 993.4653313159943 failure_reason: 'Test failure: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread)' flavor: default mon.a-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 mon.b-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=faf45a1ab6264ac29c1d0f6e1a9cfd6d status: fail success: false 2023-11-24T06:30:55.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:30:55.427 INFO:teuthology.run:FAIL